PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting Over-refusal in Safety-aligned Large Language Models: A Representation and Optimization Perspective

Chao Chen, Pengyang Shao, Wei Qin, Shijie Hao, Fei Shen, Richang Hong

arXiv:2610.39802v1cs.AI

TL;DR

Safety-aligned LLMs can mistakenly reject harmless but superficially risky queries, and why this persists under correct safety supervision remains insufficiently understood. The paper analyzes representation and gradient patterns and proposes GOAL, which combines gradient orthogonalization with contrastive learning. Experiments report mitigation of over-refusal while maintaining safety performance and general capabilities.

  • Problem

    Why safety-aligned LLMs still reject harmless queries under correct safety supervision remains insufficiently understood despite existing mitigation methods.

    Source: §1 · p. 1

  • Method

    GOAL orthogonalizes over-refusal gradients against an SVD-derived low-rank refusal subspace from harmful gradients and uses contrastive learning to separate harmful and over-refusal representations.

    Source: §1 · p. 2

  • Results

    Experiments across multiple benchmarks report that GOAL mitigates over-refusal while maintaining safety performance and general capabilities.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper’s analysis links refusal-target gradient coupling and reduced representational separation with possible difficulty maintaining a clear harmful–over-refusal boundary.

    Source: §1 · pp. 1–2

    Evaluation focuses on open-weight 7–13B instruction-tuned models and predominantly English benchmarks; larger, proprietary, and multilingual models need further evaluation.

    Source: Limitations and Ethical Considerations · p. 10

Abstract

from arXiv · show

Over-refusal is a critical side effect of safety alignment in large language models (LLMs), causing harmless but superficially risky queries to be mistakenly rejected. Despite the effectiveness of existing mitigation methods, why safety-aligned LLMs may still reject harmless queries under correct safety supervision remains insufficiently understood. To address this gap, we analyze over-refusal through the lens of representation and optimization. We make two observations: 1) the representations of harmful and over-refusal samples are visually distinguishable at certain intermediate layers but increasingly overlap in deeper layers; and 2) harmful-sample gradients exhibit high intra-class similarity and concentrate in a low-rank subspace. Moreover, they are more strongly aligned with refusal-target gradients than with correct helpful-target gradients on over-refusal prompts. This imbalanced gradient coupling can bias over-refusal samples toward refusal, while the reduced apparent representational separation in deeper layers may make it more difficult for the model to maintain a clear boundary between harmful and over-refusal samples. To this end, we propose GOAL (Gradient Orthogonalization with contrAstive Learning), a safety-alignment framework for mitigating over-refusal. Specifically, GOAL consists of two components: 1) to prevent imbalanced gradient coupling from biasing the optimization of over-refusal samples toward refusal, we orthogonalize over-refusal gradients against the low-rank refusal subspace extracted from harmful gradients; and 2) to explicitly enhance the separation between harmful and over-refusal representations, we introduce a contrastive learning objective. Extensive experiments across multiple models and benchmarks demonstrate that GOAL effectively mitigates over-refusal while preserving safety performance and general capabilities.

1 Introduction

The introduction frames over-refusal as the mistaken rejection of harmless but superficially risky queries and identifies an explanatory gap in prior mitigation work. It analyzes representational and gradient patterns, proposes GOAL, and reports gains over the second-best methods.

1 Introduction

  • Prior mitigation methods reshape refusal boundaries or intervene in safety-critical layers, but why correct safety supervision still yields over-refusal remains insufficiently understood.DCR uses contrastive refinement before safety alignment; SCANS steers activations with refusal-related directions.Source: §1 · p. 1
  • Harmful and over-refusal representations are distinguishable at layer 13 but increasingly overlap in deeper layers, potentially making their boundary harder to maintain.Figure 1 visualizes layer-wise last-token representations for benign, harmful, and over-refusal samples in Llama-2-7B-Chat.Source: §1 · p. 1, Figure 1 · p. 2
  • Harmful gradients show high within-class similarity and concentrated low-rank structure, while over-refusal gradients couple more strongly with refusal-target than helpful-target gradients.The gradient analyses compare cumulative explained ratios and pairwise similarities across sample types.Source: §1 · pp. 1–2, Figure 2 · p. 2
  • These patterns suggest that gradient coupling may bias over-refusal samples toward refusal, while reduced representational separation may hinder a clear harmful–over-refusal boundary.The proposed explanation is framed as a possibility, not as the sole cause of over-refusal.Source: §1 · pp. 1–2
  • GOAL extracts a low-rank refusal subspace from harmful-sample gradients with SVD, orthogonalizes over-refusal gradients against it, and adds contrastive learning to separate representations.Both components are applied while fine-tuning selected safety-relevant layers.Source: §1 · p. 2

B SVD extracts the refusal subspace

  • GOAL achieves up to 14.03% relative gains in over-refusal mitigation and 1.20% in safety performance over the second-best methods.The paper also reports that it maintains general capabilities.Source: B SVD extracts the refusal subspace · p. 3

2 Related Works

Prior work studies over-refusal through dedicated false-refusal benchmarks and mitigation methods, but offers limited explanation for its persistence under correct safety labels. Related optimization research covers boundary shifts, a perspective rarely applied to over-refusal.

2.1 Over-refusal in Safety-aligned LLMs

  • Safety alignment can reduce harmful outputs while causing models to reject benign requests that appear potentially unsafe, and dedicated benchmarks assess these false refusals.Prior mitigation methods reshape refusal boundaries or steer activations in safety-critical layers, but provide limited explanation for over-refusal under correct labels.Source: §2.1 · p. 3

2.2 Decision Boundary Shifts in Optimization

  • Research on decision-boundary shifts examines margin-based optimization, contrastive or metric learning, and biased or imbalanced training signals.These perspectives address changes in inter-class separation, representation geometry, or learned boundaries.Source: §2.2 · p. 3
  • Over-refusal resembles a boundary shift in which harmless but safety-sensitive samples move toward the refusal side, yet this perspective has rarely been used to analyze it.Source: §2.2 · p. 3

3 Methodology

GOAL combines refusal-orthogonal projection and safety-aware contrastive learning to address optimization-level coupling and reduced representational separation. Its design extracts dominant refusal directions from harmful-sample gradients, projects over-refusal updates, and separates harmful and over-refusal representations.

3.1.1 Identifying the Low-Rank Refusal Subspace.

  • Harmful-sample gradients concentrate in a low-rank subspace, so GOAL uses SVD to retain top directions as an orthonormal refusal basis.The paper selects k directions to capture dominant refusal-gradient energy.Source: §3.1.1 · p. 4

3.1.2 Why Over-refusal Samples Are Attracted to the Refusal Sub-

  • Harmful gradients show high intra-class similarity and stronger coupling to refusal-target than helpful-target gradients on over-refusal samples, potentially increasing refusal preference.The paper formalizes this effect: when the refusal-subspace contribution exceeds the residual contribution in magnitude, a sufficiently small learning rate increases the refusal preference margin.Source: §3.1.2 · p. 4

3.1.3 Refusal-orthogonal Projection for Decoupled Optimization.

  • GOAL projects available helpful-target gradients for over-refusal samples away from the refusal subspace while preserving harmful gradients, avoiding weakened safety supervision.The projected update remains a first-order descent direction for over-refusal loss and removes the refusal-subspace component.Source: §3.1.3 · pp. 4–5

3.2 Safety-aware Contrastive Learning

  • A Circle-style margin contrastive loss separates harmful and over-refusal representations using final-layer hidden states and a class-balanced memory bank.The bank supplies same-label positive and different-label negative pairs, with adaptive pair weights.Source: §3.2 · p. 5

3.3.1 Why contrastive learning alone is insufficient.

  • The two components address complementary problems: projection directly controls update directions, while contrastive learning improves representation-level separation.The paper notes that contrastive learning alone does not directly control SFT gradients, which can remain biased toward refusal.Source: §3.3.1 · p. 5, §3.2 · p. 5

3 Methodology

  • GOAL achieves leading over-refusal performance while maintaining strong safety and preserving general capabilities across the reported benchmarks.Table 1 compares over-refusal, safety, and general-capability benchmarks across three backbone models.Source: Table 1 · p. 6

3.3.2 Choice of Safety-relevant Layers.

  • GOAL selects the contiguous layer block where benign–harmful separability grows fastest and restricts LoRA fine-tuning to that safety-relevant block.Contrastive loss uses final-layer representations, with gradients backpropagated to trainable LoRA parameters in the selected layers.Source: §3.3.2 · p. 6

3.3.3 Time Complexity Analysis.

  • GOAL’s added costs are truncated SVD at O(Ndk), projection at O(B_o k d), and memory-bank comparison at O(B_c|M|d_h); the paper describes overhead as modest when k and |M| are small.Here, N is the harmful-sample count, d the trainable-parameter count, and |M| the memory-bank size.Source: §3.3.3 · p. 6

4 Experiments

Across three instruction-tuned models, GOAL mitigates over-refusal while maintaining safety and general capabilities. Experiments also test its components, layer selection, refusal-subspace robustness, and hyperparameter sensitivity.

4.1 Experimental Settings

  • The experiments evaluate Gemma-7B-it, Llama-2-7B-Chat, and Llama-2-13B-Chat using a 500-sample training set of harmful, benign, and over-refusal queries.Evaluation covers over-refusal, safety, and general-capability benchmarks; models are fine-tuned with LoRA on selected safety-relevant layers.Source: §4.1 · p. 7

4.2 Overall Performance (RQ1)

  • GOAL achieves the best or tied-best performance on 13 of 15 over-refusal settings, ranks second on two, and improves Llama-2-13B-Chat OKTest from 0.9400 to 0.9700 and CoCoNot from 0.8496 to 0.8892.These comparisons are against the strongest non-GOAL results on the two OOD benchmarks.Source: §4.2 · p. 7
  • On Gemma-7B-it, GOAL improves JBB from 0.90 to 0.97 and STL from 0.9748 to 0.9884 versus the base model; its largest general-capability decrease is 0.0071.On Llama-2-13B-Chat, it matches the best STL score of 0.9981 and has a competitive JBB score of 0.97.Source: §4.2 · p. 7

4.3 Ablation Study (RQ2)

  • The full GOAL framework consistently achieves the highest Compliance Rate across over-refusal benchmarks, outperforming either component alone while maintaining competitive safety and general-capability performance.The authors characterize gradient orthogonalization and contrastive learning as complementary.Source: §4.3 · p. 7

4.4 Effectiveness of Safety-relevant Layers (RQ3)

  • On Llama-2-7B-Chat, the probe-selected middle-layer block outperforms early- and late-layer variants across over-refusal benchmarks and performs competitively with full-layer tuning using roughly one-third of its parameters.The experiments compare the middle-layer block with early-, late-, and full-layer configurations.Source: §4.4 · pp. 7–8, Table 3 · p. 8

4.5 Robustness of the Refusal Subspace (RQ4)

  • On Llama-2-7B-Chat, Compliance Rate and Safety Score remain stable as the number of harmful prompts used to construct the refusal subspace varies.The authors interpret this stability as evidence that the extracted subspace captures an intrinsic refusal structure rather than sample-specific artifacts.Source: §4.5 · p. 8, Figure 4 · p. 8

4.6 Hyperparameter Analysis (RQ5)

  • Across a broad sweep of projection rank and contrastive weight on Llama-2-7B-Chat, performance remains stable, with strong results around the default k=10 and λ=0.1.Nearby settings achieve comparable results on OKTest Compliance Rate and STL Safety Score.Source: §4.6 · p. 8, Figure 5 · p. 8

5 Conclusion

The paper analyzes over-refusal through representation and optimization, then proposes GOAL to address both observed patterns. Experiments show mitigation while preserving safety and general capabilities.

5 Conclusion

  • GOAL combines refusal-orthogonal projection to decouple over-refusal optimization from harmful gradients with safety-aware contrastive learning to separate harmful and over-refusal representations.The paper reports that harmful representations and over-refusal representations increasingly overlap in deeper layers, while harmful gradients concentrate in a low-rank subspace and align more strongly with refusal-target than helpful-target gradients on over-refusal prompts.Source: §5 · p. 9

Limitations and Ethical Considerations

The authors bound their evaluation scope and identify limits on causal interpretation and deployment claims. The appendices also prove properties of the refusal-orthogonal update and describe safety-relevant layer selection.

Limitations and Ethical Considerations

  • Experiments focus on open-weight 7–13-billion-parameter instruction-tuned models and predominantly English benchmarks; larger, proprietary, and multilingual models need further evaluation.The authors also note that strong benchmark performance does not guarantee safety in all deployment settings.Source: Limitations and Ethical Considerations · p. 10
  • The first-order analysis characterizes harmful-gradient coupling sufficiently but does not establish it as the sole cause of over-refusal.Automatic evaluation also partly relies on an LLM judge.Source: Limitations and Ethical Considerations · p. 10
  • Three master’s students independently assessed model responses; they gave informed consent, could discontinue without penalty, and received CNY 100 per hour.No personally identifying information was collected, and only aggregate results are reported.Source: Limitations and Ethical Considerations · p. 10

A Proof of Refusal-subspace Attraction

  • For a sufficiently small learning rate, the harmful-gradient update increases an over-refusal prompt’s refusal preference margin, pulling it toward refusal.The proof uses a first-order Taylor expansion and a sufficiently small learning rate.Source: A Proof of Refusal-subspace Attraction · p. 10

B Proof of Decoupling by Refusal-orthogonal Projection

  • Refusal-orthogonal projection removes the harmful-gradient component inside the refusal subspace while remaining a first-order descent direction for the over-refusal training loss.The projected gradient’s remaining overlap with the harmful gradient comes from components outside the refusal subspace.Source: B Proof of Decoupling by… · p. 11

C Choice of Safety-relevant Layers

  • Safety-relevant layers are selected by finding where the benign–harmful separation gap grows most rapidly, using within-class and between-class cosine similarities.The layer-selection probe uses benign and harmful samples as reference classes rather than over-refusal samples.Source: C Choice of Safety-relevant Layers · p. 11

D Detailed Experimental Settings

The detailed settings describe the evaluated models, datasets, benchmarks, and GOAL implementation. Additional Llama-2-13B-Chat visualizations show harmful and over-refusal representations separating at layer 14 but overlapping more in deeper layers.

D Detailed Experimental Settings

  • GOAL is evaluated on Gemma-7B-it and two Llama-2-Chat models, trained with harmful, benign, and over-refusal samples and evaluated on over-refusal, safety, and multiple-choice capability benchmarks.The 100 over-refusal prompts have superficially unsafe forms but benign underlying intent; training and evaluation sets are separate from the diagnostic sets used for representation and gradient analyses.Source: D Detailed Experimental Settings · pp. 11–12
  • Implementation uses rank-16 LoRA on selected layer subsets, a refusal-subspace rank of 10, and last-layer contrastive loss with weight 0.1.The contrastive representation bank has 128 entries, split into two class-specific FIFO queues of 64; harmful and over-refusal representations form the contrastive classes.Source: D Detailed Experimental Settings · p. 12

E Additional Layer-wise Representation Visualization

F Detailed Results for Robustness and Hyperparameter Analysis

The analyses examine GOAL’s robustness, its effect on representations, training overhead, and behavior on harmful and over-refusal queries.

F Detailed Results for Robustness and Hyperparameter Analysis

G Representation Separation after Training

  • After fine-tuning, GOAL yielded clearer visual separation between harmful and over-refusal representations across the examined middle-to-late layers.The authors describe this observation as consistent with the contrastive objective’s intended effect.Source: Figure 10 · p. 14, G Representation Separation after… · p. 15

H Training Procedure and Runtime Overhead

  • On Llama-2-7B-Chat, GOAL’s online training took 9.55 minutes versus 9.53 minutes for standard SFT, a 0.21% online overhead.Including 2.17 minutes for one-time refusal-subspace extraction, the first run took 11.72 minutes, 23.0% above standard SFT.Source: H Training Procedure and Runtime… · p. 15

I Some Qualitative Examples

  • In qualitative examples, GOAL refused a genuinely harmful query while responding helpfully to a benign query containing superficially sensitive keywords.The baseline refused the benign query.Source: I Some Qualitative Examples · p. 15

J Further Analysis of the Effectiveness of Safety-relevant Layers

This analysis compares target-layer choices and specifies a diagnostic study of the refusal-subspace attraction condition, including its setup and evaluation measures.

J Further Analysis of the Effectiveness of Safety-relevant Layers

  • Across all five over-refusal benchmarks, the complete safety-relevant layer block outperformed targeting only the separable intermediate layer or subsequent overlap layers.The table caption states that higher Compliance Rate indicates better over-refusal mitigation.Source: J Further Analysis of the… · pp. 15–16, Table 5 · p. 16

K Empirical Verification of Refusal-Subspace Attraction

  • The diagnostic tests whether Proposition 1’s sufficient condition occurs across samples, checkpoints, seeds, and backbones, without assuming it holds universally.It also tests whether the predicted first-order margin change is associated with the change after a finite-step harmful-gradient update.Source: K Empirical Verification of… · p. 16

K.1 Evaluation Setup

  • The study uses three backbones, held-out sets of 200 harmful and 200 over-refusal prompts, five epoch checkpoints, and five random seeds.Diagnostic prompts are excluded from both safety-alignment training and refusal-subspace construction.Source: §K.1 · p. 16

K.2 Evaluation Metrics

  • The metrics quantify refusal-subspace energy, condition satisfaction, median condition ratio, and the association between predicted and observed finite-step margin changes.The finite-step diagnostic uses the raw aggregated gradient and restores the original checkpoint afterward.Source: §K.2 · p. 17

K.3 Results

The results show that the proposed attraction condition is observed across models and training, while projection’s margin effects are examined theoretically and empirically.

K.3 Results

  • 0.82–0.89 of aggregated harmful-gradient energy lay in the extracted refusal subspace, and the sufficient condition held for 70.3%–83.7% of evaluated samples.Median condition ratios ranged from 1.16 to 1.25; predicted and finite-step margin changes had positive Spearman correlations of 0.24–0.37. The condition is sufficient, not necessary, and does not establish harmful-gradient coupling as the sole cause of over-refusal.Source: §K.3 · p. 17

L Refusal-Margin Reduction

  • Under a verifiable gradient-coupling condition, refusal-orthogonal projection reduces the first-order increase in refusal preference margin relative to the corresponding unprojected update.The result is relative and does not require the margin to decrease absolutely after every update.Source: L Refusal-Margin Reduction · p. 17, §L.1 · p. 18

L.1 Relative Reduction of the Refusal Preference Margin

  • Condition (50) requires the helpful-target gradient, within the refusal subspace, to couple more strongly with the refusal-target gradient than with itself.Under this condition, removing the refusal-subspace component reduces the first-order refusal-margin increase.Source: §L.1 · p. 18

L.2 Empirical Analysis of Refusal Preference Margins

  • The empirical margin analysis compares held-out over-refusal samples from the base model, vanilla SFT, and Ortho only, with projection as the sole difference between the trained variants.The comparison defines margin reduction relative to vanilla SFT to isolate the additional effect of refusal-orthogonal projection.Source: §L.2 · p. 19

L.3 Empirical Verification of the Projection Condition

  • The Proposition 3 condition held for a clear majority of evaluated samples and was moderately associated with sample-level refusal-margin reduction, but did not hold uniformly.Median condition ratios ranged from 1.12 to 1.21, and the Spearman correlations ranged from 0.24 to 0.33.Source: §L.3 · p. 20

M Robustness under AdamW Preconditioning

These analyses test projection under AdamW, control for shared refusal-target effects on gradient concentration, and evaluate response quality with an automated judge and human annotators.

M Robustness under AdamW Preconditioning

  • Ortho only reduced refusal preference margins beyond SFT by 0.055, 0.025, and 0.032 on Gemma-7B-it, Llama-2-7B-Chat, and Llama-2-13B-Chat, respectively.After AdamW preconditioning, residual refusal-subspace leakage was 3.6%–4.7%, while coupling reduction was 84.1%–88.5%.Source: M Robustness under AdamW Preconditioning · p. 21

N Controlling for Refusal-Target Artifacts

  • Harmful+Refusal had a top-10 cumulative explained ratio of 0.78, above Benign+Refusal at 0.57, and its low-rank structure persisted after centering.The results indicate that shared refusal targets account for only part of the gradient concentration.Source: Table 9 · p. 22, N Controlling for Refusal-Target… · p. 22

O Human Evaluation and Response-Quality Analysis

  • The response-quality analysis tests whether reduced refusal corresponds to useful responses and whether GPT-5.1 judgments align broadly with human judgments.It evaluates helpfulness, correctness, and completeness alongside unwarranted refusal.Source: O Human Evaluation and… · p. 22, §O.1 · p. 22

O.1 Evaluation Setup

  • The evaluation randomly samples 200 held-out over-refusal prompts and compares vanilla SFT, ACTOR, and GOAL under the same decoding configuration.Useful Compliance requires no unwarranted refusal and helpfulness and correctness scores of at least 4.Source: §O.1 · p. 22

O.2 GPT-5.1 Quality Evaluation

  • GPT-5.1 evaluates unwarranted refusal and assigns 1-to-5 scores for each response-quality dimension using a fixed prompt and deterministic decoding.Method identities are hidden from the evaluator.Source: §O.2 · pp. 22–23

O.3 Human Evaluation

  • Three human annotators independently assess each prompt-response pair; non-refusal requires unanimous agreement, and quality ratings are averaged.Across 600 pairs, agreement was Fleiss’ κ=0.71 and ordinal Krippendorff’s α=0.75–0.81.Source: §O.3 · p. 23

O.4 Results

GOAL has the lowest reported unwarranted-refusal rates under both GPT-5.1 and human evaluation, while also receiving the strongest response-quality scores. The evaluations agree that fewer refusals are accompanied by useful responses.

O.4 Results

  • GPT-5.1 refusal rates were 19.5% for SFT, 14.5% for ACTOR, and 9.0% for GOAL; human rates were 21.5%, 15.0%, and 10.0%, respectively.GOAL had the lowest unwarranted-refusal rate under both evaluation procedures.Source: §O.4 · p. 23
  • Under human evaluation, GOAL’s Useful Compliance rate was 76.0%, versus 66.5% for SFT and 70.0% for ACTOR.GOAL also achieved the highest helpfulness, correctness, and completeness scores under both evaluation procedures.Source: §O.4 · p. 23
  • GPT-5.1 and human evaluation both rank GOAL first across all reported metrics, with similar refusal-rate and response-quality trends.The authors present this agreement as evidence for the reliability of GPT-5.1-based evaluation and for genuinely useful responses accompanying the refusal reduction.Source: §O.4 · p. 23

P Multi-Seed Evaluation and Statistical Significance

Across five-seed evaluations, GOAL generally outperforms ACTOR on over-refusal benchmarks while maintaining safety scores. Comparisons also favor low-rank harmful-gradient projection over alternative interventions, whereas downweighting harmful gradients entails a safety trade-off.

P Multi-Seed Evaluation and Statistical Significance

  • The comparison uses five matched-seed runs and hierarchical paired bootstrap resampling, with Holm–Bonferroni correction across five over-refusal benchmarks per backbone.The safety benchmarks are reported with means and sample standard deviations, without significance testing.Source: P Multi-Seed Evaluation and… · p. 23
  • Across three backbones, GOAL maintains an overall advantage across seeds, with statistically significant gains over ACTOR on most over-refusal benchmarks and safety scores above 0.96 on JBB and 0.98 on STL.Exceptions include a nonsignificant PHT result for Llama-2-7B-Chat and a slightly higher ACTOR mean on Gemma-7B-it ORH; safety significance was not assessed.Source: P Multi-Seed Evaluation and… · pp. 23–24, Table 11 · p. 24

Q Comparison with Alternative Gradient Intervention Strategies

  • PCGrad is less effective than Ortho only across all five over-refusal benchmarks, while the strategies achieve comparable safety performance.The authors note that refusal-subspace coupling need not appear as conventional negative interference between aggregated gradients; Ortho only scores 0.95 versus PCGrad’s 0.96 on JBB and 0.9806 versus 0.9767 on STL.Source: Q Comparison with Alternative Gradient… · pp. 24–25, Table 12 · p. 25
  • Ortho only raises average compliance rate from 0.8616 for Harmful Mean Ortho to 0.8738 and outperforms it across all five over-refusal benchmarks.This supports representing harmful-gradient structure with multiple dominant singular directions rather than a single mean direction.Source: Q Comparison with Alternative Gradient… · pp. 24–25, Table 12 · p. 25

R Effect of Harmful-Gradient Downweighting

  • Reducing the harmful-objective weight to 0.5 improves compliance on PHT, OKT, and CoCo but lowers JBB and STL safety scores to 0.86 and 0.8779.Ortho only achieves comparable overall over-refusal mitigation while scoring 0.95 on JBB and 0.9806 on STL.Source: R Effect of Harmful-Gradient… · p. 25, Table 13 · p. 25

S Evaluation on a Recent Model Family

On Qwen3-8B, GOAL improves over-refusal compliance while largely preserving safety and general capabilities. Separately, harmful and over-refusal representations become less separable in deeper layers, and controls indicate that projection effectiveness depends on the harmful-sample subspace.

S Evaluation on a Recent Model Family

  • The Qwen3-8B evaluation uses five independent runs and includes over-refusal, safety, and general-capability benchmarks.Qwen3-8B differs from the main-experiment models in architecture, tokenizer, and post-training recipe.Source: S Evaluation on a Recent Model Family · p. 25, Table 14 · p. 26
  • On Qwen3-8B, GOAL achieves the highest mean compliance rate on four of five over-refusal benchmarks and ranks second on ORH.Compared with SFT, compliance-rate gains are 1.78, 0.16, 2.34, 2.33, and 1.06 percentage points on XST, ORH, PHT, OKT, and CoCo, respectively.Source: S Evaluation on a Recent Model Family · p. 25, Table 14 · p. 26
  • GOAL scores 0.98 on JBB and 0.9953 on STL, while achieving accuracies of 0.7597, 0.5600, 0.8107, and 0.5121 on MMLU, OpenBookQA, PIQA, and ARC-Challenge.The authors report that general-capability differences from the base model and SFT are small on the first three benchmarks, and GOAL outperforms ACTOR on all four.Source: S Evaluation on a Recent Model Family · pp. 25–26, Table 14 · p. 26

T Quantitative Analysis of Representation Separation

  • Harmful and over-refusal representations are most separable at layer 13 (AUC 0.858; separation ratio 0.357), declining to 0.678 and 0.207 at layer 29.Probe AUC remains above the 0.5 chance level, so the distinction weakens rather than disappears.Source: T Quantitative Analysis of… · p. 26, Table 15 · p. 26
  • Harmful Ortho achieves the best compliance rate on four of five benchmarks, while Random Ortho and Benign Ortho underperform SFT across all five.The comparison indicates that projection effectiveness depends on identifying the dominant subspace induced by harmful samples with refusal supervision.Source: T Quantitative Analysis of… · p. 26, Table 16 · p. 26
Loading 2610.39802v1…