PaperlayerView exact revision on arXiv

Source-linked AI summary

Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs

Zixuan Weng, Haoqi He, Weifei Jin, Yue Huang, Yang Bai

arXiv:2610.13078v1cs.AI

TL;DR

Safety-aligned LLMs can comply with jailbreaks or refuse benign requests, and behavioral outcomes alone do not show whether harmfulness was missed or refusal writing failed. The paper measures this recognition-to-writing pathway in residual-stream updates and uses its findings to build Sentinel. Attention and MLP contributions differ across the pathway, while joint-signal conditional routing achieves lower aggregate error than the evaluated single-signal and fixed-reminder alternatives.

  • Problem

    Behavioral benchmarks capture jailbreaks and over-refusals but do not reveal whether a failure reflects unrecognized harmfulness or recognized harmfulness that fails to produce refusal.

    Source: §1 · p. 1

  • Method

    The paper measures harmfulness signals in attention updates and refusal writing in MLP updates, then combines these as perception score P and decision score D in Sentinel.

    Source: §1 · p. 2

  • Results

    Attention-side harmfulness recognition and MLP-side refusal writing form a pathway strengthened by post-training, with writing concentrated late and at answer onset; joint-signal routing lowers aggregate error versus single-signal routing and evaluated fixed reminders.

    Source: §1 · p. 2, §5 · pp. 9–10

  • Takeaways & Limitations

    The findings suggest strengthening refusal writing rather than harmfulness representations alone within the studied pathway.

    Source: §1 · p. 2

Abstract

from arXiv · show

Safety-aligned large language models (LLMs) remain vulnerable to jailbreaks and can over-refuse benign requests. Understanding these failures requires distinguishing harmfulness recognition from the computation that writes refusal. We reframe this as a pathway question and make it measurable on residual-stream updates in decoder-only Transformers, separating the harmfulness information a module update exposes from the change it produces in readable refusal evidence. Across 18 Base-Instruct pairs spanning Qwen, Llama, and Gemma, harmfulness is decoded more readily from attention updates, particularly in early layers, whereas MLP updates contribute more strongly to harm-selective refusal writing in aligned models. Post-training strengthens MLP refusal writing in all 18 pairs without systematically improving harmfulness decodability, while positive MLP writing becomes more concentrated in deeper layers after post-training. Across six reasoning-enabled models, harmfulness remains decodable during reasoning while strong refusal writing concentrates at answer onset, a pattern that persists with reasoning disabled. Causal interventions further support this pathway: resetting downstream MLP updates to their clean values substantially reduces the refusal shift caused by upstream attention interventions. Building on these findings, we introduce Sentinel, which combines an attention-derived perception score P with an MLP-derived decision score D. Their joint profiles characterize jailbreaks and over-refusals, and conditional routing with both signals achieves a lower aggregate error rate than single-signal routing and evaluated fixed reminders across eight models.

1 INTRODUCTION

The paper asks how Transformers turn harmfulness recognition into refusal, distinguishing recognition from the computation that writes refusal. It traces this pathway and develops Sentinel, finding different module contributions across post-training and generation and lower aggregate error with joint-signal routing than with evaluated alternatives.

1 INTRODUCTION

  • Behavioral benchmarks show jailbreaks and over-refusals but do not identify whether failures reflect missed harmfulness or failure to turn recognition into refusal.Prior mechanistic work identifies safety-relevant representations and interventions, but the paper frames the recognition-to-refusal transformation as the open pathway question.Source: §1 · pp. 1–2
  • Across 18 Base–Instruct pairs, attention updates expose stronger harmfulness signals, especially early, while MLP updates dominate harm-selective refusal writing in post-trained models.Restoring clean downstream MLP updates substantially weakens the refusal-readout shift caused by upstream attention interventions.Source: §1 · p. 2
  • Post-training strengthens harm-selective MLP writing without systematically increasing harmfulness decodability; positive MLP writing also becomes more concentrated in deeper layers.Across six reasoning-enabled models, harmfulness remains decodable during reasoning while strong refusal writing concentrates at answer onset.Source: §1 · p. 2
  • Sentinel combines an attention-derived perception score P with an MLP-derived decision score D to characterize jailbreaks and over-refusals and route conditional reminders.Across eight models, joint routing achieves lower aggregate error than single-signal routing and the tested fixed reminders.Source: §1 · p. 2
  • The paper concludes that safety alignment is a recognition-to-writing pathway and suggests strengthening refusal writing rather than harmfulness representations alone.Its contributions include testing the functional link with causal interventions and using the account to route reminders before decoding.Source: §1 · p. 2

2 RESIDUAL-STREAM ANALYSIS OF HARMFULNESS AND REFUSAL

The section defines a residual-stream framework that separates harmfulness information in module updates from their effects on refusal evidence and harmfulness separability. It specifies measures for recognition, refusal writing, and selectivity, then previews how these quantities are visualized.

2 RESIDUAL-STREAM ANALYSIS OF HARMFULNESS AND REFUSAL

  • The framework distinguishes harmfulness information linearly decodable from a module update, refusal evidence readable from a residual state, and the update-induced change in that evidence.Comparing the refusal-evidence change across harmful and harmless prompts measures selectivity for harmful inputs.Source: §2 · p. 2

2.1 RESIDUAL STATES AND MODULE UPDATES

  • Each Transformer block is represented as an attention update followed by an MLP update added to the residual stream.The residual state is measured at a token position entering a layer; answer_start is the position immediately before the first visible answer token.Source: §2.1 · p. 3

2.2 MEASURING HARMFULNESS RECOGNITION AND REFUSAL WRITING

  • Probes on attention or MLP updates measure harmfulness decodability, while separability gain measures whether adding an update improves residual-stream harmfulness separability.The measures can differ: an informative update may add little separability when its information is already present in the residual state.Source: §2.2 · p. 3
  • The refusal-evidence measures use residual-state readouts and compare the evidence before and after a module update; positive writing increases refusal evidence and negative writing decreases it.The section also defines a harm-selective gap by comparing group-averaged writing across harmful and harmless prompts.Source: §2.2 · pp. 3–4
  • The figures organize the measurements by layer or generation position, including cross-model medians, interquartile ranges, and normalized writing profiles.The normalized profiles show writing shape and location, not absolute magnitude.Source: Figure 1 · p. 4, Figure 2 · p. 4, Figure 3 · p. 5

3 TRACING THE RECOGNITION-TO-WRITING PATHWAY

Across the studied models, harmfulness is more readable from attention updates, while late MLP updates contribute strongly to refusal writing. Post-training strengthens and concentrates that writing, and during generation it appears most strongly at answer onset rather than accumulating through reasoning.

3.1 WHICH MODULE EXPOSES HARMFULNESS AND WRITES REFUSAL EVIDENCE?

  • Attention has higher layer-averaged harmfulness-probe AUC than MLP in all 36 checkpoints, with the largest gap in early layers.Attention also produces positive residual-stream separability increments concentrated early, whereas MLP increments are near zero or slightly negative despite decodable harmfulness information in MLP updates.Source: §3.1 · p. 6
  • In all 18 Instruct checkpoints, MLP net writing exceeds attention net writing, with the largest positive MLP contributions near the network’s end; downstream MLP restoration substantially weakens upstream attention patching effects.The measured organization localizes harmfulness separability mainly to early attention and refusal writing mainly to late MLPs.Source: §3.1 · p. 6

3.2 HOW DOES POST-TRAINING RESHAPE THE PATHWAY?

  • Post-training produces no systematic increase in harmfulness probe AUC, but MLP net refusal writing increases in all 18 Base–Instruct pairs.MLP writing rises in 18/18 pairs, compared with attention writing in 12/18 pairs.Source: §3.2 · p. 6
  • After post-training, MLP refusal writing becomes more directional and less canceled: MLP specificity rises from 1.10 to 4.20 and its negative-to-positive writing ratio falls from 0.78 to 0.32.Attention specificity rises from 0.77 to 1.63, while its cancellation ratio falls from 1.66 to 1.20 and remains above one.Source: §3.2 · p. 6
  • The final quarter’s share of positive MLP writing rises from a median 23.01% before post-training to 63.86% after post-training, with Thinking models at 66.13%.The increase in writing strength persists in all six tested matched-input pairs, although the exact depth profile varies across models.Source: §3.2 · p. 7

3.3 WHERE DOES REFUSAL WRITING EMERGE DURING GENERATION?

  • Across six models, harmfulness remains readable throughout reasoning, varying in strength, while with reasoning enabled MLP refusal writing stays weak without accumulating at reasoning positions, then jumps at answer_start to median 16.76.With reasoning disabled, answer-boundary medians are 13.94 for MLP and −1.47 for attention, and MLP writing exceeds attention in all six models. Probe AUC is approximately U-shaped, falling during reasoning and recovering at the answer boundary.Source: §3.3 · p. 7

4 SE N T I N E L: PATHWAY DIAGNOSIS AND CONDITIONAL REMINDERS

Sentinel combines attention-derived harmfulness perception with MLP-derived refusal writing to diagnose jailbreak and over-refusal profiles and route reminders. Its category analysis distinguishes error patterns, and conditional reminders improve the reported balance of safety and helpfulness outcomes.

4.1 PERCEPTION AND DECISION SCORES

  • P aggregates attention-derived harmfulness-recognition signals, while D aggregates MLP-derived refusal-writing scores for prompt-level diagnosis.The scores are used for category analysis; routing retains selected layerwise features.Source: §4.1 · pp. 7–8

4.2 DIAGNOSING PROMPT CATEGORIES

  • P/D reference profiles place normally answered benign prompts at low P and D and correctly refused harmful prompts at high P and D.Figure 5 shows model-standardized group means; panel axis ranges vary, and zero is not a behavioral threshold.Source: §4.2 · p. 8, Figure 5 · p. 8
  • ReNeLLM shows the lowest P and relatively low D alongside the highest ASR, whereas PAIR and AutoDAN retain higher P but weak refusal writing and high ASR.GCG’s refusal writing is close to AdvBench and it has lower ASR.Source: §4.2 · p. 8
  • Over-refused XSTest-safe prompts have higher mean P and D than normally answered prompts, while their P remains below AdvBench; EVOREFUSE-TEST shows the same direction.On EVOREFUSE-TEST, over-refused subsets have higher D in all three models.Source: §4.2 · p. 8

4.3 SENTINEL-GUIDED CONDITIONAL REMINDERS

  • Conditional routing uses calibrated P/D features after prefill to classify request behavior and select no reminder, a safety reminder, a helpfulness reminder, or both.Evaluation spans eight models with request-grouped nested cross-validation and reports jailbreak ASR plus false-refusal rates for benign and previously over-refused requests.Source: §4.3 · p. 9
  • Sentinel lowers jailbreak ASR (95.97% to 35.17%) and over-refusal false refusal (88.73% to 55.72%) versus no reminder, while benign false refusal rises from 0.00% to 0.13%.The fixed Safety reminder has 30.45% jailbreak ASR but 85.24% over-refusal; the Helpfulness reminder has 59.95% over-refusal but 68.22% jailbreak ASR.Source: §4.3 · p. 9, Table 1 · p. 9
  • Using both signals yields a 30.34% AVG, below P-only routing at 38.09% and D-only routing at 34.96%.The paper attributes their usefulness to complementary information: P captures harmful intent, while D reflects refusal tendency.Source: §4.3 · p. 9

5 CONCLUSION

The conclusion frames safety alignment as an attention-to-MLP pathway and presents Sentinel as a diagnostic and routing application of its two signals.

5 CONCLUSION

  • The paper frames safety alignment as attention-side harmfulness recognition followed by MLP-side refusal writing, with post-training strengthening MLP writing but not systematically improving harmfulness decodability.Strong refusal writing concentrates in late layers and at answer onset, with or without reasoning.Source: §5 · p. 9
  • Restoring downstream MLP updates to clean values reduces part of the refusal shift caused by upstream attention interventions, supporting the proposed pathway.Sentinel combines both signals to diagnose failures and guide conditional reminders, achieving lower aggregate error than single-signal routing and evaluated fixed reminders.Source: §5 · p. 10

AI USE STATEMENT

The authors disclose generative-AI assistance in manuscript preparation and an LLM-based evaluator in the experimental methodology.

AI USE STATEMENT

  • Generative AI assisted with language editing, writing refinement, and LaTeX-related tasks; an LLM-based evaluator judged jailbreak outcomes.The authors state that they reviewed and verified the scientific claims, experimental designs, analyses, results, and final text.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The study uses public safety benchmarks, excludes human subjects and private user data, and acknowledges potential misuse of mechanistic analyses.

ETHICS STATEMENT

  • Experiments use publicly available safety and over-refusal benchmarks and involve neither human subjects nor private user data.The authors note that mechanistic analyses of safety failures could potentially be misused to weaken safeguards.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The paper documents materials and procedures for reproducing its analyses, then situates Sentinel among behavioral benchmarks and internal refusal-pathway research.

REPRODUCIBILITY STATEMENT

  • The main paper and Appendices B–H provide model inventories, datasets, measurement definitions, intervention protocols, evaluation prompts, and analysis procedures.These materials support reproduction of the representation, intervention, and Sentinel-routing analyses.Source: REPRODUCIBILITY STATEMENT · p. 10

A RELATED WORK

  • Sentinel uses refusal benchmarks and jailbreak families to map four behavioral groups—benign answering, correct refusal, jailbreak success, and over-refusal—to locations in the P–D plane.The related-work discussion connects this diagnosis to refusal-control and localization studies that examine attention, MLP updates, and residual-state changes.Source: A RELATED WORK · p. 15

B.1 MODULE-UPDATE COUNTERFACTUALS SUPPORT DISTINCT ATTENTION AND MLP ROLES

Module-update counterfactuals distinguish Attention’s upstream influence from MLP refusal writing, and restoration experiments test how much downstream MLP updates mediate upstream Attention effects.

B.1 MODULE-UPDATE COUNTERFACTUALS SUPPORT DISTINCT ATTENTION AND MLP ROLES

  • Attention patching generally shifts final refusal evidence in the expected direction, but its aggregate same-layer MLP response is comparatively small and inconsistent.The authors interpret this as a pathway-level upstream effect, not a strict layer-by-layer causal chain.Source: §B.1 · pp. 16–17, Figure 6 · p. 16
  • MLP patching generally moves final refusal evidence and same-layer MLP writing in aligned directions, especially in later layers.This provides more direct causal support for MLP updates as refusal-evidence writers.Source: Figure 6 · p. 16, §B.1 · p. 17
  • At reasoning answer boundaries, the same local–final distinction appears: Attention patching has a comparatively small same-layer MLP response, while MLP patching aligns local writing and final refusal shifts.The analysis summarizes 10 reasoning configurations.Source: §B.1 · p. 17, Figure 7 · p. 17

B.2 DOWNSTREAM MLP RESTORATION BLOCKS UPSTREAM ATTENTION EFFECTS

  • Clean-MLP restoration has a nonzero blocking effect in 13 of 14 model–mode-by-direction cells, with blocking fractions ranging from 11% to 87% across reported non-reasoning configurations.Qwen3.5-9B NR is lower at 19% and 11%; the Llama-3.1-8B benign-to-harmful cell is the exception with an inconclusive total effect.Source: §B.2 · p. 18, Figure 9 · p. 19
  • 26.5–82.8% of the upstream effect is removed by MLP restoration, versus −34.6% to 19.2% by matched-site Attention restoration in the held-out comparison.The paired MLP-minus-Attention difference is positive in all three models, with 95% bootstrap intervals excluding zero in all six model–direction comparisons.Source: §B.2 · pp. 18–19, Table 2 · p. 20
  • Selected downstream MLP restoration blocks more than fixed random post-P restoration in all five model–direction comparisons with total-effect intervals excluding zero.The paired intervals for the selected-minus-random blocking difference are above zero in each comparison.Source: §B.2 · p. 20, Table 3 · p. 20
  • In generation outcomes, benign-target refusals change from 11→39→14 for Qwen3-8B NR and 14→58→16 for Gemma4-12B NR across CLEAN, P-PATCH, and P-PATCH+D-RESTORE.For Gemma4-12B NR, harmful-target compliance changes from 4→22→5; restoration is weaker in reasoning and Qwen3.5 configurations.Source: Figure 10 · p. 19, §B.2 · p. 20

C ROBUSTNESS TO THE REFUSAL-READOUT VOCABULARY

Across alternative refusal-readout vocabularies, the paper’s principal temporal, module-level, and routing patterns persist, supporting robustness beyond one manually specified token set.

C.2 WHEN DOES REFUSAL-RELATED NET WRITING INTENSIFY?

C.3 DOES ANSWER-BOUNDARY ENHANCEMENT DEPEND ON GENERATED REASONING?

  • Cumulative MLP writing increases from assistant_start to answer_start in all eight configurations, with gains of +2.58 to +19.97 thinking-off and +7.80 to +24.15 thinking-on.The gains hold across all four readouts; positive gains under thinking-off show generated reasoning is not necessary for answer-boundary enhancement.Source: Figure 12 · p. 21, §C.3 · pp. 24–25

C.4 DO ALTERNATIVE READOUTS PRESERVE THE LATE-LAYER MLP WRITING STRUCTURE?

  • Across all 12 model–readout configurations, the Instruct–Base MLP layer-sum difference is positive, and MLP net writing exceeds Attention net writing in 11.In 11/12 configurations the largest positive single-layer MLP contribution occurs in the final layer; the sole module-comparison exception is Qwen3-8B with data-driven k = 8.Source: Figure 13 · p. 22, C ROBUSTNESS TO THE REFUSAL-READOUT… · p. 22, §C.4 · p. 25

C.1 CONSTRUCTION OF ALTERNATIVE READOUTS AND CALIBRATION ISOLATION

  • The alternative readouts use refusal-prefix tokens or prompt-disjoint calibration generations to construct data-driven refusal and compliance token sets.Candidates must occur at least five times and achieve directional consistency of at least 0.95 across 2,000 bootstrap replicates; the k = 3 set is nested within k = 8.Source: §C.1 · pp. 22–23

C.5 DO ALTERNATIVE READOUTS PRESERVE DOWNSTREAM ROUTING?

  • Routing AVG across the four readouts ranges from 33.37% to 35.45%, with alternative-readout changes of +2.08, +0.56, and +0.26 percentage points from the main token set.These are three-model macro averages and are not directly comparable to the eight-model averages in Table 1.Source: §C.5 · pp. 25–26, Table 8 · p. 26

C.6 SUMMARY

  • All four readouts reproduce the principal structures, so the reported temporal and module-level patterns do not depend on one manually specified vocabulary or refusal-opening token.The summary also reports preservation of conditional-reminder routing across readout constructions.Source: §C.6 · p. 26

D MATCHED-INPUT AND MATCHED-POSITION CONTROL FOR BASE–INSTRUCT COMPARISONS

Matched-input, matched-position controls show that post-training strengthens MLP refusal writing across all six tested checkpoint pairs, while its depth profile remains context-sensitive. The supplied routing passages describe how Sentinel features, classifiers, and actions are evaluated, including a low-dimensional comparison of joint and single-signal routers.

D MATCHED-INPUT AND MATCHED-POSITION CONTROL FOR BASE–INSTRUCT COMPARISONS

  • The control compares six Base–Instruct pairs on the same 100 harmful and 100 matched benign prompts, using a shared plain-text cue and measurement position.The models receive identical token IDs; MLP refusal writing is measured at the final colon immediately before the first answer token.Source: D MATCHED-INPUT AND MATCHED-POSITION… · p. 26
  • MLP net writing remains higher after post-training in all six matched-input, matched-position pairs.This increase persists when Base and Instruct checkpoints receive the same token sequence and are measured at the same position.Source: D MATCHED-INPUT AND MATCHED-POSITION… · pp. 27–28
  • Under matched inputs and positions, the late-quarter positive-mass share increases in 2/6 pairs, while peak depth shifts later in three, stays fixed in one, and moves earlier in two.The precise depth profile varies across models, making writing strength the more consistent post-training effect.Source: D MATCHED-INPUT AND MATCHED-POSITION… · pp. 27–28

E.1 EVALUATED MODELS AND DATASET CONSTRUCTION

  • The routing evaluation covers eight instruction-tuned models from the Llama, Qwen, and Gemma families, with up to 500 unique requests per model across six cohorts.The routing pool contains 3,662 requests; four-action evaluation uses requests with outputs available for every action.Source: §E.1 · pp. 28–29

E.2 P/D FEATURES

  • The main router uses selected layerwise P and D features extracted after prompt prefill and before response generation, with model-specific layers and training-split standardization.P and D windows contain 2–7 and 1–12 layers, respectively.Source: §E.2 · p. 29

E.3.1 OUTER AND INNER FOLDS

  • Evaluation uses grouped nested cross-validation, keeping prompts derived from the same underlying request in the same fold.The eight non-reasoning models use five outer folds and three inner folds.Source: §E.3.1 · p. 29

E.3.2 FOUR-CLASS ROUTER

  • Sentinel predicts four behavioral states and maps their probabilities to reminder actions, with candidate classifiers, weighting schemes, and routing rules selected on training folds.Inner-fold selection minimizes the equal-weight average of jailbreak ASR and over-refusal error subject to feasibility constraints; held-out predictions are aggregated across five outer splits.Source: §E.3.2 · p. 29, §E.3.3 · p. 30, §E.3.4 · p. 30

E.4 ROUTING WITH LOW-DIMENSIONAL P/D FEATURES

  • The low-dimensional routing comparison averages selected layerwise values into one P scalar and one D scalar, then compares P-only, D-only, and joint P+D routers.These variants use the same models, requests, cached reminder outputs, behavioral judgments, and grouped nested cross-validation protocol.Source: §E.4 · p. 30

E.4.1 RESULT

The low-dimensional joint P+D router has lower aggregate error than either single-signal router, with a positive joint advantage in six of eight models. Sentinel layer selection uses layerwise evidence to identify attention perception windows and sustained post-perception MLP writing windows.

E.4.1 RESULT

  • 39.34% AVG for the 2D joint router compares with 42.96% for P alone and 41.36% for D alone; its individual error rates are no higher than either single-signal router.The full layerwise router achieves 30.34% AVG, lower than all three window-averaged variants.Source: §E.4.1 · p. 30
  • Six of eight models have lower AVG with the joint router, with a mean gain of 1.50 percentage points and a median gain of 0.62 points.The paired model-level bootstrap 95% percentile interval is [0.28, 3.10] points.Source: §E.4.1 · p. 31
  • Gemma4-31B-Instruct falls back to no reminder in every outer fold because no low-dimensional candidate meets feasibility constraints, while Qwen3-14B-Instruct has ∆m = −0.50 points.The joint advantage therefore does not hold uniformly across models.Source: §E.4.1 · p. 31

F AUTOMATIC LAYER SELECTION FOR SENTINEL SCORES

  • Automatic P-layer selection retains layers whose one-sided 95% lower confidence bound for attention-probe AUROC exceeds chance, then selects a high-performing contiguous plateau.For MLP decision layers, selection targets the earliest sustained positive-writing component after the P window; if none is eligible, Qwen3-14B uses the component with the largest integrated positive writing gap.Source: F AUTOMATIC LAYER SELECTION FOR… · pp. 31–32

G.1 MODELS AND EXPERIMENTAL SCOPE

The paper separates model inventories and evaluation settings for representation analysis, interventions, and Sentinel routing. It describes judge-based safety metrics, paired and benchmark reference groups, and refusal/compliance token sets used for refusal-evidence readouts.

G.1 MODELS AND EXPERIMENTAL SCOPE

  • The representation analysis covers 18 Base–Instruct pairs, while reasoning-position analysis uses six Qwen3/Qwen3.5 checkpoints with thinking enabled and paired thinking-off runs.The single-module counterfactual analysis has 31 model–mode configurations; the joint path-cutting experiment is narrower, with seven configurations.Source: §G.1 · pp. 33–34

G.2 ASR EVALUATION.

  • Attack success rate is judged from whether a response provides actionable unsafe assistance, rather than from surface-level refusal wording.The prompt-based judge classifies responses as reject or jailbreak.Source: §G.2 · p. 34

G.3 OVER-REFUSAL EVALUATION.

  • Over-refusal is a judge’s refuse classification on an allowed request, while reasonable caveats do not count as refusal when substantive assistance is still provided.Source: §G.3 · p. 35

G.4 BENCHMARKS AND ATTACK GROUPS

  • JailbreakBench supplies closely matched harmful and benign prompts for pathway analysis, while XSTest and EvoRefuse provide over-refusal reference groups.The matched prompts help reduce surface-form, length, and style confounds; XSTest OR and EvoRefuse refused prompts support over-refusal comparisons.Source: §G.4 · p. 36
  • The four analyzed jailbreak groups are ReNeLLM, PAIR, AutoDAN, and GCG, which differ in how they alter or optimize harmful prompts.ReNeLLM rewrites requests to appear benign, PAIR iteratively refines prompts with an attacker model, AutoDAN optimizes readable adversarial prompts, and GCG searches for adversarial suffixes.Source: §G.4 · pp. 36–37

G.5 REFUSAL AND COMPLIANCE TOKEN SETS.

  • Refusal and compliance token sets are used in the log-sum-exp refusal evidence readout and are held constant across models for comparability.Matched-control specificity uses 16 size-matched vocabularies, including controls matched by unembedding-row-norm quantile.Source: §G.5 · p. 37

H PER-CONFIGURATION RESULTS

The per-configuration figures show module-update counterfactuals for non-reasoning configurations and at the answer boundary after reasoning, separating local MLP writing from final refusal evidence.

H PER-CONFIGURATION RESULTS

  • Figure 16 covers 21 non-reasoning configurations, with attention and MLP patching in separate columns and final refusal evidence and same-layer MLP writing in separate panels.Red and blue distinguish harmful-donor-to-benign-target from benign-donor-to-harmful-target patching.Source: Figure 16 · p. 40
  • Figure 17 shows the corresponding single-update counterfactuals for 10 reasoning configurations, intervening at answer_start after cached reasoning context.It uses the same column, panel, and patch-direction encodings as Figure 16.Source: Figure 17 · p. 51
  • Across the per-configuration display, compare how patching attention versus MLP updates changes final refusal evidence and local MLP writing in each inference setting.The inventory table identifies the model configurations included in the single-module analysis.Source: Figure 16 · p. 40, Figure 17 · p. 51, Table 17 · p. 56
Loading 2610.13078v1…