Source-linked AI summary
Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration
Yiqi Liu, Yang Wang, Songxin Wang, Chenghao Xiao, Chenghua Lin
TL;DR
The paper asks whether refusal erases an answer internally or suppresses its release, and tests this using matched causal interventions in a controlled withhold setting. It finds that answer release is highly local, whereas suppression and coherent refusal assembly require broader interventions, revealing broken symmetry.
Problem
It is unclear whether refusal is a symmetric, local switch, because directional steerability does not establish equally local inverse control over answer release and coherent refusal.
Method
The study compares matched answering and refusal trajectories in an instruction-forced A/B withhold task using bidirectional residual-stream activation patching across layers and positions.
Results
Answer release requires a single-position patch, whereas suppression requires broader multi-position interventions and coherent refusal assembly is harder; a shared displacement vector is not a reversible behavioural toggle.
Takeaways & Limitations
Probe recoverability and local steering can overestimate behavioural control, so refusal audits should assess answer release, suppression, full-refusal generation, and locality gaps.
Takeaways & Limitations
The evidence primarily comes from an idealized instruction-forced multiple-choice withhold setting, so naturalistic prevalence and task-induced asymmetry remain unevaluated.
Abstract
from arXiv · showhide
When a language model refuses to answer a prompt, it is unclear whether the correct answer is erased from its internal representations, or merely suppressed at the output layer. We investigate this mechanism using a controlled withhold setting, which yields perfectly matched answering and refusal trajectories for bidirectional activation patching. We uncover a causal asymmetry in intervention locality under matched causal interventions, which we term broken symmetry. Even when a model generates a clean refusal, the correct answer remains linearly recoverable from its hidden states. Furthermore, releasing this withheld answer is a highly local operation, requiring only a single-position patch. Conversely, the reverse operation is not equally local: reimposing suppression requires broader interventions across multiple positions, and assembling a coherent refusal sequence is more difficult still. We further demonstrate that while an average answer-to-refusal displacement vector marks the geometric difference between these states, it fails to act as a reliable, reversible linear control toggle between behaviours. Taken together, our findings show that refusal does not function as a simple symmetric switch. For safety and auditing, this implies that probe recoverability can overestimate true behavioural control, and locating refusal-relevant directions does not reliably grant the ability to steer a model from answering to coherent refusal.
1 Introduction
The paper tests whether refusal is a local, symmetric internal switch and finds broken symmetry: releasing a withheld answer is highly local, whereas restoring suppression and coherent refusal requires broader intervention. Hidden answers remain recoverable, but refusal geometry does not reliably provide reversible behavioural control.
- Suppression and assembly: Suppressing the correct answer channel is causally distinct from assembling a coherent refusal string under intervention.The paper treats local answer suppression and explicit, stable refusal generation as mechanistically different behaviours.
- Directional control: An average answer-to-refusal displacement vector does not provide a reliable, reversible toggle between answering and coherent refusal.The shared direction marks a geometric difference between states but does not establish causal symmetry or matched inverse edits.
- Intervention locality: A single-position patch restores the withheld answer, whereas reimposing answer-channel suppression requires broader patches across multiple positions.This asymmetry is established through bidirectional activation patching under matched causal interventions.
- Recoverability and control: The correct answer remains linearly recoverable during a clean refusal, so probe-only safety audits can overestimate behavioural control.Recoverability of hidden information does not establish that interventions can reliably change the model’s generated behaviour.
- Validation: Across 7B-to-32B models from Qwen, Llama, Mistral, and Gemma, hidden-state asymmetry persists while generation-level and shared-direction control remain checkpoint-dependent.The qualitative asymmetry is robust across model sizes and families, but intervention strength varies by checkpoint.
2 Related Work
Prior work links causal patching, decoder-restricted probing, hidden knowledge, refusal geometry, and activation steering to the study of suppressed information and safety-relevant behavior. This paper extends these lines by testing whether answer release and refusal restoration are equally local and whether refusal-relevant directions can assemble full behavior.
- Causal patching and decoder-restricted recovery: Causal activation patching builds on causal mediation and tracing, while decoder-restricted probing treats the decoder family as part of the measured quantity.These foundations motivate using progressively richer decoder families alongside causal patching.
- Hidden knowledge and refusal geometry: Models can encode information internally despite suppressing it in their outputs, and refusal studies identify shared directions and multidimensional concept cones.These works characterize refusal-relevant geometry but do not test whether answer release and refusal restoration are mirror-local inverse operations.
- Activation steering and safety mechanisms: Activation addition, sparse-autoencoder feature localization, over-refusal mitigation, and weight pruning demonstrate directional steerability for safety-related behavior.Prior methods do not ask whether a refusal-relevant direction is sufficient for full behavior assembly; this paper addresses that question with matched bidirectional patching.
3 Experimental Setup
The experiments use a controlled alternate-choice withhold task to create matched answering and refusing trajectories for bidirectional activation patching. They evaluate hidden-answer recoverability, local answer-channel suppression, and generation-level refusal assembly across models and patch windows.
- Models and the Withhold Task: Qwen2.5-7B-Instruct is the primary testbed, with scaling checks on Qwen2.5-14B/32B and cross-family checks on Llama, Mistral, and Gemma checkpoints.The primary model is open-weight, instruction-tuned, stable under the controlled withhold gate, and suitable for high-resolution activation patching.
- Models and the Withhold Task: The controlled alternate-choice task asks the model to hide the correct A/B option while preserving a fixed answer channel, alongside Greater-Than and IOI baselines.The answering and refusing prompts differ by an explicit withhold instruction, while the underlying multiple-choice question remains identical.
- Patching on Matched Trajectories: Matched trajectories have identical token lengths, enabling residual-stream state transplantation between answering and refusing runs at selected layers and token positions.Patches use cached pre-sampling residual-stream states Rℓ,t and range from position -1 to wider windows including positions -2 and -3.
- Patching on Matched Trajectories: The two causal directions test local answer release by patching A-run states into R-run states and refusal restoration by patching R-run states into A-run states.These interventions respectively test whether the hidden answer can be released locally and whether suppression and refusal assembly can be reimposed.
- Metrics and Decoders: The evaluation separates probe recoverability, answer-channel suppression, and full refusal assembly using discrete readouts, continuous margins, and generation-level refusal markers.Recoverability uses linear, MLP, affine, and tuned lenses; suppression uses A/B top-1 and correct-answer-minus-distractor margin; refusal assembly scores explicit refusal markers versus answer matching.
- Metrics and Decoders: The primary asymmetry measure is a single-position locality gap comparing late-layer A/B readout changes for A→R release versus R→A suppression.A positive gap means answer release is causally easier, or more local, than restoring suppression.
4 Results
The results show that withheld answers remain fully recoverable, but releasing them is much more local than restoring suppression or assembling coherent refusal. Bidirectional patching and vector interventions reveal that refusal-relevant geometry does not provide a reliable reversible behavioural control.
- Probe recovery: A linear probe perfectly recovers the withheld answer (1.000) from the same token that later produces a refusal.Probe-only recovery therefore motivates causal patching tests rather than establishing behavioural control.
- Bidirectional patching: Across Qwen2.5-7B layers 16–26, single-position A→R patches yield 1.00 recovery, while matched R→A patches remain at the 0.50 refusal baseline.The same final-answer-token intervention therefore releases the hidden answer but does not restore answer-channel suppression.
- Bidirectional patching: Widening R→A patches from −1 to (−1, −2, −3) reduces answer rates from 0.50 to 0.00 and raises explicit refusal from 0.00 to 0.84.Coherent refusal consequently requires distributed support across earlier context positions.
- Generalisation: The asymmetry persists across strictly gated naturalistic prompts and multiple model families, with a comparable late-layer separation of around 0.6.The reported checks include Llama-3.1-8B, Mistral-7B, Gemma-4, and larger Qwen checkpoints.
- Vector interventions: The mean refusal displacement concentrates 85%, 84%, and 89% of energy at layers 16, 22, and 26, yet it does not form a reversible behavioural axis.In Qwen2.5-7B, λ = 0.5 at layer 16 reduces answer generation from 100% to 0%, whereas explicit refusal reaches 84% only at λ = 2.
5 Mechanism Analysis
Withhold-refusal suppression relies on persistent late-layer, non-adjacent context beyond ordinary answer substitution. Across model families and scales, removal-side effects recur, but concentrated shared-direction geometry does not reliably provide affine behavioural control.
- 5.1 Locality: Withhold-refusal patching retains late-layer position −2 support and a nonzero position −3 footprint, unlike ordinary substitution’s transient off-slot spillover.The forced-override control’s position −2 spillover vanishes by late layers and is zero at position −3, whereas refusal retains both footprints.
- 5.1 Locality: 0.672 of cases released a clean answer after a layer-16 position −2 generation patch, showing that this support is not merely a readout artifact.Refusal nevertheless continues to rely on non-adjacent context late in the forward pass.
- 5.1 Locality: Single-head, attention-only, and MLP-only patches at layer 16 across positions −1, −2, and −3 were behaviourally null in both directions.These controls argue against a simplest sparse-carrier account but do not provide a full circuit decomposition.
- 5.2 Cross-model scope: Removal-side effects recur in Llama-3.1-8B and Mistral-7B, while Gemma-4 shows stronger removal than refusal-assembly effects and larger Qwen models lack comparable direction-intervention leverage.Hidden-state patching still recovers the release-suppression asymmetry at Qwen2.5-14B and Qwen2.5-32B; the main evidence remains Qwen-centric.
6 Discussion
The discussion characterizes broken symmetry as a recurring causal asymmetry: answer release is more local than suppression, while coherent refusal assembly and reversible displacement control remain unreliable. This asymmetry limits probe-only auditing and local steering as measures of behavioural control.
- Safety auditing: Probe recoverability can remain high during answer withholding, so probe-only audits overestimate behavioural deployability.Refusal audits should jointly assess probe recoverability, answer release, answer-channel suppression, and full-refusal generation.
- Safety auditing: Local steering success can overstate refusal control because answer-channel suppression and full-refusal assembly remain only partially aligned under intervention.Locating refusal-relevant structure does not by itself specify how refusal behaviour is causally controlled.
- Cross-model checks: Cross-family checks support the qualitative asymmetry, while larger-Qwen checks add constraints on the displacement-based mechanism claim.Removal-side interventions can release answers or degrade refusal, whereas addition-side interventions do not reliably assemble coherent refusal.
- Interpretation and future work: The proposed explanation is that sequence-level alignment may install distributed refusal policies while leaving local answer representations comparatively intact.Testing this hypothesis requires controlled comparison of SFT-only and SFT+RL variants of the same base model.
- Implications: Across tested settings, answer release is more local than suppression, suppression is not equivalent to coherent refusal assembly, and displacement is not generally reversible.The average answer-to-refusal displacement does not generally provide a reversible coordinate between the two behaviours.
Limitations
The evidence is centered on an idealized, instruction-forced multiple-choice withhold setting, while naturalistic checks and cross-family coverage remain limited. The experiments establish intervention-locality findings without providing a full circuit decomposition or testing the post-training mechanism hypothesis.
- Controlled setting: The main evidence comes from an instruction-forced multiple-choice withhold setting that fixes the answer space, instruction contrast, and scoring rule.Its claims concern asymmetric intervention locality in controlled refusal trajectories, leaving naturalistic prevalence and task-induced asymmetry for separate evaluation.
- Naturalistic checks: Naturalistic free-form checks are weaker, heterogeneous causal stress tests on retained examples rather than prevalence estimates over harmful prompts.The larger stress test is speech-heavy, while email and article forms were excluded because harmful-side refusal was unstable under the same gate.
- Model coverage: Cross-family coverage is partial and does not support family-general claims, with Qwen2.5-7B as the main experimental line and other families providing supporting profiles.The prompt format, gate criteria, and layer-band conventions were developed on Qwen2.5-7B.
- Model coverage: Differences across model families may reflect pipeline coupling as well as genuine mechanistic variation because the experimental conventions were developed on Qwen2.5-7B.This limitation applies to the prompt format, gate criteria, and layer-band conventions.
- Mechanistic scope: The experiments do not provide a circuit decomposition or identify the full attention-mediated route sustaining refusal token by token.The post-training mechanism hypothesis remains untested and would require a controlled SFT-only versus SFT+RL comparison.
Ethical Considerations … B.1 Out-of-Set Override Controls
The paper frames its findings as diagnostic and dual-use: hidden-state recoverability can coexist with broader behavioral requirements for answer deployment or suppression. Its appendices specify the information-theoretic motivation, evaluation templates and scoring rules, supporting refusal tables, and an out-of-set override control.
- Ethical Considerations: Refusal audits should not collapse local recoverability and actual behavior into one score, because deployment or suppression may require broader distributed support.Probe-only or local-state-only audits can therefore overestimate safety under perturbation.
- Ethical Considerations: Recoverable hidden-state information is dual-use: stronger decoders or causal interventions could recover or amplify sensitive information that is behaviorally suppressed.Local steering directions may appear more behaviorally decisive than they are because answer suppression and full refusal assembly are only partially coupled.
- A Information-Theoretic Motivation and Bound Sketches: The appendix motivates the empirical analysis with hidden states Rℓ,t and a task label T, while treating its derivation as background rather than a separate theoretical contribution.The reported experiments were run on NVIDIA GPUs, using single L40S-class GPUs for lighter runs and A100-class GPUs or equivalent sharded execution for larger follow-ups.
- A Information-Theoretic Motivation and Bound Sketches: Decoder-restricted quantities provide lower bounds whose tightness improves with richer nested decoder families, motivating comparisons among linear, MLP, affine-lens, and tuned-lens decoders.The paper does not claim that any one decoder fully characterizes the underlying representation.
- A Information-Theoretic Motivation and Bound Sketches: A high linear-probe value is used only to establish simple-decoder recoverability; behavioral claims instead rely on matched causal patching.The empirical paper does not depend on treating decoder-restricted bounds as tight estimates of mutual information.
- A.1 Main Prompt Templates and Scoring Rules: The controlled evaluation pairs answer and refusal prompts with identical questions and option order, using exact answer and refusal templates.The refusal template instructs the model to withhold the answer and output exactly REFUSE.
- A.1 Main Prompt Templates and Scoring Rules: Scoring combines patched-logit top-1 reads, correct-answer-minus-distractor margins, generation-level answer matching, and fixed-marker refusal detection.Answer matching requires the decoded continuation to begin with the stripped correct label token, A or B.
- B Refusal Supporting Tables for Qwen2.5-7B: The Qwen2.5-7B supporting tables include main-setting refusal results, matched control analyses, and an out-of-set override control testing whether answer release persists beyond the original A/B set.The out-of-set control specifically places the prescribed override target outside the original answer set.
B.2 Sampling Robustness · C Cross-Model Removal-Side Evidence and Geometry Comparison
The merged sections test whether answer release remains local under non-greedy sampling and distinguish cross-model removal-side causal evidence from geometric concentration. Across the primary setting, broader patches strengthen both behavioral directions, while cross-model evidence is kept separate from geometry/control comparisons.
- B.2 Sampling Robustness: Non-greedy decoding is used to test whether generation-level answer release persists beyond greedy sampling.The section explicitly frames sampling robustness around generation-level release under non-greedy decoding.
- B.2 Sampling Robustness: In the main withhold setting, single-position late-layer answer→refuse patching releases the hidden answer locally, whereas refuse→answer degrades answering without reinstating refusal.The generation-level locality gap survives into greedy generation, while stronger suppression requires wider patches across earlier positions.
- B.2 Sampling Robustness: Qwen2.5-7B sampling experiments use 10 samples per pair to measure answer-channel release under single-position patching.Table 10 defines these rows as answer-channel release measurements.
- B.2 Sampling Robustness: At layer 16, refuse→answer answer rate falls from 0.50 to 0.17 and then 0.00 as patches widen from −1 to (−1, −2) and (−1, −2, −3).The corresponding refusal rate rises from 0.00 as the patch widens, showing stronger behavioral suppression with broader interventions.
- B.2 Sampling Robustness: Sparse interventions produce no behavioral change in either direction, while full-layer patching reproduces the behavioral asymmetry.The reported sparse controls include single-head, attention-only, and MLP-only interventions at layer 16 across positions (−1, −2, −3).
- C Cross-Model Removal-Side Evidence and Geometry Comparison: Cross-model evidence is organized outside the primary setting and separates larger-model geometry/control contrasts from cross-family removal-side causal evidence.This separation is intended to avoid conflating removal-side causal evidence with geometric concentration.
C.1 Within-Family Contrast: Qwen2.5-14B … E.1 Generation-Boundary Positions
Across models, hidden-state answer release remains more reliable and local than refusal restoration, while shared-direction geometry and generation-level control weaken with scale and vary across families. The analyses also delimit the mechanism’s scope, expose coordinate instability, and locate non-adjacent support at generation-boundary positions.
- C.1 Within-Family Contrast: Qwen2.5-14B: Qwen2.5-14B shows narrow hidden asymmetry: answer release reaches 1.00 only at layer 26, while reverse suppression remains 0.00 across layers 16, 22, and 26.Generation-level single-position patching is inert in both directions, and shared-direction interventions are behaviorally inert at tested late layers.
- C.1 Within-Family Contrast: Qwen2.5-14B: Qwen2.5-32B preserves hidden asymmetry at depth: answer→refuse patching reaches 1.00 at layers 40, 48, 54, and 58, while refuse→answer suppression remains 0.00.At layers 16, 22, and 26, bidirectional patching was fully inert, indicating a graded rather than binary scaling pattern.
- C.2 Cross-Family Supporting Evidence: Llama-3.1-8B and Mistral-7B: Llama-3.1-8B and Mistral-7B support the removal-side constraint: matched removal collapses refusal and raises answer generation, whereas addition lacks comparably broad stable control.Llama reverse edits do not reliably reinstall explicit refusal strings, while Mistral addition can saturate at one tested layer without forming a stable late-layer coordinate.
- C.3 Gemma-4 Profiles: Gemma-4-E4B and Gemma-4-26B-A4B show near-maximal asymmetry: removal or release fully collapses refusal and restores answering, while addition fails to assemble explicit refusal.These profiles clarify the removal-versus-addition distinction rather than establish one shared refusal mechanism across families.
- D Qwen2.5-3B Transparency: Coordinate-System Instability: Qwen2.5-3B is reserved for transparency because chat-template geometry is less one-dimensional and the raw refusal-side baseline is unstable.Removal-style follow-ups can produce answer behaviour without defining a clean refusal-side counterpart to the principal Qwen2.5-7B/Qwen2.5-14B assay.
- E Scope and Robustness Analyses: The scope analyses characterize non-adjacent support and test whether wider suppression automatically improves refusal assembly.These analyses constrain the paper’s main claims rather than introduce a broader mechanism claim.
- E.1 Generation-Boundary Positions: In Qwen2.5-7B and Llama-3.2-3B, strongest non-adjacent effects at positions −2 and −3 fall on structural generation-boundary tokens, not simple lexical cues.This supports locating non-adjacent support at generation boundaries rather than reducing it to keyword anchoring.
E.2 Wider-Patch Follow-Up
Widening the suppressive refuse→answer patch from (−1, −2, −3) to (−1, −2, −3, −4) did not improve refusal assembly and exposed an interaction between patch width and position identity. Because −4 lies just outside the assistant-generation boundary and no full skip-position search was run, the result is a boundary check rather than a systematic explanation of the reversal.
- Wider-patch follow-up: At layer 16, extending the patch to (−1, −2, −3, −4) restored answer to 1.00 and refusal to 0.00, fully reversing the earlier gain.The wider patch did not improve refusal assembly.
- Wider-patch follow-up: At layer 22, the wider patch still suppressed answers without reinstating strong refusal.This indicates that wider patching was not uniformly beneficial across layers.
- Interpretation and limitation: The −4 slot lies just outside the assistant-generation boundary, and no full combinatorial search over skip-position patches such as (−1, −2, −4) was run.The table should therefore be read as a boundary check, not a systematic explanation of the −4 reversal.
F Naturalistic Refusal Checks
Naturalistic checks retained stable answer-versus-refusal artifact families and found the same directional asymmetry as the controlled setting, with answer release broad across late layers but refusal reimposition concentrated early. A speech/thread robustness expansion preserved this pattern despite an uneven retained set, while the naturalistic endpoint supports the finding directionally rather than on the controlled experiment’s absolute scale.
- Artifact screening: Email and article artifacts were excluded because harmful-side refusal remained unstable under the patching gate.The pipeline first established a clean retained set, then expanded only artifact families with stable answer-versus-refusal baselines.
- Artifact screening: The retained 66-case set supports generation-level causal analysis, but speech prompts are more stable than thread prompts and blog prompts are too sparse for a separate claim.This set is an operational diagnostic rather than a balanced artifact sample.
- Main naturalistic check: 0.64/0.62/0.59 operational-answer release remains broad across layers 16–26, while refusal reimposition is front-loaded at layer 16 at 0.45 and attenuates to 0.02 by layer 22.The naturalistic generation endpoint differs from the controlled hidden-state locality comparison, so only the sign and direction of the gap are compared.
- Robustness check: The retained set expanded from 66 to 283 cases, with 253 speech and 30 thread examples, yet the aggregate causal pattern remained stable under the stronger speech bias.The expansion used speech and thread prompts only.
- Robustness check: 0.67/0.70/0.66 operational-answer release remains broad across layers 16–26, whereas refusal reimposition is front-loaded at layer 16 at 0.51.The expanded speech-heavy check is directionally consistent with the main naturalistic set, with refusal reimposition near-zero by layers 22–26.