Source-linked AI summary
A Unified Mechanistic Analysis of Knowledge- and Safety-Based Refusals
Yuri Son, Seunghee Kim, Hyuhng Joon Kim, Taeuk Kim
TL;DR
KR and SR produce similar refusals despite different epistemic and normative grounds, but their shared mechanism has remained unclear. The paper addresses this gap with matched contrastive quadruples and controlled training, finding shared refusal commitment, asymmetric transfer, and late type-specific specialization.
Problem
KR and SR have largely been studied in isolation, leaving whether they share an internal mechanism unresolved.
Method
The paper uses matched contrastive quadruples and controlled sequential training to analyze shared and type-specific refusal representations.
Results
KR and SR share a refusal direction, but SR signals transfer more strongly to KR and type-specific specialization emerges in upper layers.
Takeaways & Limitations
Refusal is best characterized as a shared initial commitment followed by type-specific epistemic or normative specification.
Takeaways & Limitations
The analysis is formulated for open-weight models, while closed-source extension is left for future work.
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly trained to decline queries that fall outside their knowledge (knowledge-based refusal, KR) or violate safety policies (safety-based refusal, SR). Although KR and SR result in superficially similar responses, they have largely been studied in isolation, leaving open whether they share an underlying mechanism. We address this gap with a systematic study on a new dataset of 213 contrastive quadruples that jointly probe both refusal types. We find that KR and SR are governed by overlapping yet distinguishable mechanisms. Both share a refusal direction, yet the overlap is asymmetric: SR signals transfer more strongly to KR than the reverse. Type-specific specialization emerges mainly in upper layers, with KR aligning with uncertainty- and knowledge-related representations and SR with safety- and policy-related ones. We thus characterize refusal as a commit-then-specify process: a shared initial mechanism commits to refusing, then type-specific features in later layers specify whether the grounds are epistemic or normative.
1 Introduction
KR and SR arise from different epistemic and normative conditions but produce similar refusal behavior, motivating a unified mechanistic study. Using matched contrastive data and controlled training, the paper finds shared and type-specific refusal mechanisms.
- KR reflects epistemic limitations, whereas SR reflects adherence to safety and usage guidelines.
- Because KR and SR were largely studied in isolation, their shared internal mechanism and transfer relationship remained unresolved.
- The study introduces 213 contrastive quadruples pairing KR and SR refusals with matched knowledge and safety controls.
- The matched design isolates refusal-specific behavior, while sequential training first on KR and then SR balances refusal capability with general performance.
- KR and SR share a refusal direction but remain distinguishable, with SR signals transferring more strongly to KR and specialization emerging in upper layers.
- Refusal follows a commit-then-specify process: a shared mechanism commits to refusal, followed by type-specific grounds.
2 Related Work
Prior work treats KR and SR as separate refusal problems and has provided mechanistic evidence mainly for SR. This separation leaves their relationship and shared inner workings unclear, motivating matched contrastive analysis.
- KR research emphasizes epistemic abstention, while SR research emphasizes harmful, unethical, or policy-violating requests.
- KR and SR have been evaluated under distinct objectives and metrics, with little attention to their relationship.
- Existing mechanistic studies identify refusal-related directions in SR representations and show that manipulating them modulates refusal behavior.
- Most mechanistic analyses focus only on SR, leaving whether KR shares the same inner workings unresolved.
- Direct KR–SR comparison is hindered by differences in prompt content, triggering cues, and model training.
3 Controlled Experimental Setup
The experimental setup uses matched KR, SR, and control prompts to isolate refusal-specific signals, alongside unified sequential tuning across model families. The selected KR-then-SR schedule preserves general utility while expressing both refusal types.
- Existing KR and SR benchmarks differ in prompt structure, semantic domain, and surface form, confounding direct comparison.
- The dataset contains 213 quadruples sharing topic and format, with KR–KC and SR–SC contrasts isolating refusal-specific signals.
- Matched controls minimally edit refusal triggers while preserving topic, key terms, and structure.
- The schema covers 20 topic categories and uses shared templates so KR/SR pairs differ primarily in their refusal-inducing factor.
- The construction pipeline filters candidates through rule-based and LLM-based checks, automated control validation, behavioral verification, random sampling, and human review.
- Sequential tuning injects both refusal capabilities under a unified protocol to reduce training-history confounds.
- Seq-KS, tuning knowledge before safety, preserves general capability and knowledge correctness while achieving target KR and SR refusal behavior.
4 Experiments
Experiments analyze matched refusal and control representations across layers, testing shared directions, cross-type transfer, and late specialization. They find a dominant common refusal component alongside persistent type-specific signals, asymmetric transfer, and semantic divergence in upper layers.
- 213 matched quadruples are analyzed across six models using final prompt-token hidden states and representation-level probes.The quadruples pair knowledge and safety refusals with matched controls; behavioral steering is also evaluated.
- 4.1 RQ1: Shared Refusal Directions: KR–SR refusal directions show peak-layer alignment of 0.699–0.794 across all six models, with the common directional component remaining elevated across layers.The dominant component reaches a 0.92–0.95 projection at the peak layer, while type-specific residual norms remain 0.32–0.39.
- 4.1 RQ1: Shared Refusal Directions: The common refusal component exceeds its shuffled-label null by about 0.45 AUC, compared with about 0.11 for the type-specific probe.This pattern holds across all six models and layers, supporting the dominance of the common direction.
- 4.2 RQ2: Asymmetric Cross-Type Transfer: SR→KR transfer reaches 0.80–0.86, while KR→SR transfer is 0.58–0.63, producing a 0.22–0.27 gap across every model.The gap appears from early layers and remains stable across depth; the asymmetry also persists across additional models, tuning orders, and a matched set.
- 4.3 RQ3: Late Type-Specific Specialization: Both refusal shifts remain near zero on the final-layer KR-specific direction through most layers and diverge only in the final layers.Thus, transfer asymmetry is detectable early, whereas the associated grounding-specific direction becomes explicit late.
- 4.3 RQ3: Late Type-Specific Specialization: The common direction promotes generic refusal tokens, while the KR–SR contrast separates uncertainty-related vocabulary from legal, normative, and privacy-related vocabulary.This vocabulary split indicates distinct refusal rationales during generation.
5 Discussion
The findings support a commit-then-specify account: a shared mechanism initiates refusal, while later type-specific grounding determines whether its basis is epistemic or normative. The asymmetry also suggests that SR relies more on the shared component, whereas KR depends more on later answerability grounding.
- Commit-then-specify: A shared refusal component first commits the model to refusing, after which later layers specify epistemic uncertainty for KR or normative constraints for SR.This account explains how refusals can appear similar while having different grounds.
- Mechanistic interpretation: SR signals transfer more strongly to KR than KR signals to SR, indicating asymmetric dependence on shared and type-specific mechanisms.The asymmetry reflects relative component contributions rather than refusal strength itself.
- Practical implications: General refusal objectives may affect both refusal types, while KR may additionally require targeted calibration of uncertainty and answerability.The authors do not directly evaluate this proposed post-training strategy.
6 Conclusion
The study analyzes KR and SR with matched contrastive data and controlled training, finding a shared but asymmetric refusal direction and late-emerging type-specific features. It consequently treats refusal as a shared commitment followed by type-specific specification.
- 6 Conclusion: Matched contrastive quadruples and controlled training isolate refusal-specific signals from prompt-content and training-history confounds.The analysis targets the representational structure of knowledge- and safety-based refusals.
- 6 Conclusion: KR and SR share a dominant refusal direction, but SR-derived representations generalize to KR more readily than the reverse.This establishes overlap alongside asymmetric transfer between refusal types.
- 6 Conclusion: Type-specific features emerge sharply in final layers, with KR aligning with epistemic signals and SR with normative signals.The result supports distinguishing refusal grounds rather than treating refusal as monolithic.
Limitations
The study’s mechanistic conclusions are bounded by its open-weight focus, single-turn setting, and deliberately controlled benchmark. These choices leave closed-source models, multi-turn interactions, and less controlled refusal scenarios for future evaluation.
- Scope of analysis: The analysis method is formulated for open-weight models because their internal representations can be directly inspected.Extending it to closed-source models would require reliable output-level refusal proxies.
- Interaction setting: The study focuses on single-turn interactions, leaving the effects of prior dialogue, follow-up queries, and accumulated user information unresolved.Those contextual factors may influence subsequent refusal behavior.
- Benchmark scope: The 213-quadruple benchmark is a controlled diagnostic set rather than a comprehensive sample of real-world refusals.Its close topic and surface-form alignment may underrepresent ambiguous, borderline, or open-domain cases.
Ethical Considerations
The paper frames its refusal analysis as controlled safety research and describes a matched-prompt construction process designed to isolate refusal-specific behavior. It also emphasizes responsible use because mechanistic findings could weaken safeguards.
- Responsible use: Safety-refusal prompts are used solely to analyze refusal behavior and representations associated with different refusal types.The stated aim is understanding and evaluating refusal mechanisms.
- Responsible use: Because mechanistic refusal insights could weaken safeguards, the authors encourage responsible use for model analysis, evaluation, and safety research.This frames the work as supporting safer and more reliable language models rather than bypassing protections.
- Dataset construction: Prompt construction proceeds through schema design, candidate generation, rule filtering, quality judgment, control construction, validation, and behavioral verification.The final retained set contains 269 clean quadruples in the construction pipeline, while the paper reports 213 total quadruples for the benchmark.
- Prompt matching: The schema uses semantic categories, action-oriented frames, shared templates, and fixed anchors to preserve comparable request structure across variants.Anchors keep topical content fixed while the primary refusal-inducing factor changes.
A.6 Behavioral Verification and Human Validation
Behavioral verification retained only quadruples producing the expected refusal and non-refusal contrast across all six models. Human review checked semantic distinction, surface matching, and natural phrasing, with strong agreement between automated and human validation.
- 213 quadruples remained after excluding pairs that failed the expected refusal/non-refusal criterion on any model.The initial set contained 269 quadruples, and retained items showed the expected contrast across every analyzed model.
- Human reviewers assessed 75 quadruples for unambiguous KR/SR distinction, matched controls, and natural surface wording.They checked that refusal rationale, rather than topic or phrasing, was the substantive difference.
- κ = 0.82 indicated strong agreement between LLM-based and human validation results.
B Model Selection and Behavioral Validation
The study compared refusal-oriented checkpoints and selected sequential knowledge-then-safety tuning because it best preserved general capability while supporting both refusal behaviors. Safety-first or single-objective alternatives showed greater over-refusal, capability loss, or unresolved trade-offs.
- Model selection: Seq-KS was selected across all three model families because it most consistently preserved utility, reduced incorrect knowledge answers, and maintained selective safety refusal.Llama showed only a minor MMLU decrease, Qwen matched or exceeded its original metrics, and Gemma was the most stable.
- Behavioral validation: TriviaQA Inc. fell from 19.77 to 12.04 for Llama, 30.25 to 11.62 for Qwen, and 19.57 to 8.80 for Gemma while Corr. stayed near original levels.
- Behavioral validation: Seq-SK raised Safe Ref. to 74.80 for Llama and 84.20 for Gemma while causing sharper MMLU and GSM8K declines than Seq-KS.The pattern reflects substantial over-refusal on benign prompts under safety-first tuning.
- Candidate comparison: Knowledge-tuned checkpoints left Safe Ref. high across families, whereas safety-tuned checkpoints degraded knowledge correctness and general utility.The reported Safe Ref. values for knowledge-tuned models were Llama 59.60, Qwen 54.80, and Gemma 68.60.
- Candidate comparison: Merged training performed reasonably on safety metrics but had lower general utility and QA correctness than Seq-KS.
C Experimental Details
Experiments used matched prompt contrasts, layer-wise representation analyses, probes, steering, and component ablations across original and sequentially tuned model variants. Diagnostic procedures were paired with controls and explicit cautions about interpretation.
- Setup: Each backbone was evaluated in both its original instruction-tuned form and its corresponding sequentially tuned variant.
- Representation analysis: Control-subtracted final prompt-token shifts were used to reduce token-length and suffix-related baseline effects within matched quadruples.Refusal directions were estimated by normalized difference-in-means using KC for KR and SC for SR as matched controls.
- Probes: Cross-type probes measured transfer between refusal types, with bootstrap intervals reported as descriptive evidence rather than a standalone confirmatory test.Probes included within-type, cross-type, common-refusal, and direct type-specific configurations.
- Activation steering: Activation steering added refusal directions at a peak-alignment layer and swept intervention strength, using random-vector controls and a conservative lexical refusal classifier.
- Interpretation: The decomposition is a geometric diagnostic: orthogonal residuals and ablation effects do not by themselves establish independent causal mechanisms or a complete circuit.
- Component analysis: Late-layer MLP ablation produced larger KR-specific projection reductions than SR-specific reductions, especially beyond relative depth 0.9.
D.1 Additional Mechanistic Analyses
Additional analyses localized shared and type-specific refusal contributions across heads, probes, and MLP blocks. They support a dominant common signal alongside late, model-dependent specialization and asymmetric transfer between refusal types.
- Head-level contributions: Top common heads were concentrated in late layers for Llama and Qwen but distributed across middle and late layers for Gemma.
- Head ablation: KR-specific head ablation generally reduced KR-specific projection more than random-head ablation, while Gemma showed a positive SR shift.Common-head ablation produced smaller and less consistent selective effects.
- Cross-type transfer: SR→KR probe transfer consistently exceeded KR→SR transfer across base and tuned models.The asymmetry was qualitatively consistent across model families, although its strength and layer profile varied.
- Interpretive boundary: Head-level findings support sensitivity of diagnostic projections, not a complete causal circuit decomposition.
- Head-level contributions: Shared heads aligned with both KR and SR directions, while K-specific and S-specific heads deviated toward their respective axes.
- MLP ablation: Late-layer MLP ablations produced larger KR than SR projection reductions, complementing attention-head evidence for late KR-specific grounding.The strongest asymmetry appeared in Llama and Qwen, while Gemma showed weaker asymmetry.
- Activation steering: Activation steering effects increased near-monotonically with α up to 20 in Llama and Qwen, supporting behavioral relevance of the extracted directions.
D.2 Robustness Across Models, Tuning Orders, and Datasets
Robustness analyses indicate that the shared KR–SR refusal structure and asymmetric transfer pattern extend across additional models, tuning orders, and dataset constructions. The supplementary evaluations preserve stronger SR→KR than KR→SR transfer and support consistent refusal-direction alignment.
- Mechanistic diagnostics: Figure 14 scans zero-ablations of MLP blocks across layers and reports the asymmetry score ΔSR −ΔKR, with positive values indicating KR-biased effects.The figure marks the late-layer region as depth > 0.9.
- Across models: Two additional open-weight models retain strong KR–SR refusal-direction alignment and the asymmetric pattern in which SR→KR transfer exceeds KR→SR.The models are Qwen3.5-9B and Gemma-3-12B-it.
- Across tuning orders: Across both Seq-KS and reversed Seq-SK tuning orders, SR→KR transfer remains consistently stronger than KR→SR.Absolute transfer accuracy varies across tuning configurations, but the direction of asymmetry is unchanged across all three model families.
- Mechanistic diagnostics: Figure 15 measures refusal rate as a function of steering strength α for K- and S-directions applied to K- and S-type control prompts.Llama∗ and Qwen∗ show near-monotonic increases through α = 20, while Gemma∗ shows a weaker but positive response.
- Across datasets: A separate XSTest-derived set of 125 matched examples retains strong KR–SR refusal-direction alignment for Qwen3.5-9B.The dataset construction changes, while the reported shared refusal representation remains consistent across the two datasets.