PaperlayerView exact revision on arXiv

Source-linked AI summary

ALTSTEER: Selective Safety Steering for Moving Beyond Hard Refusals to Constructive Alternatives

Hoejoon Kwon, Byeonggeuk Lim, Kahyeon Kim, YoungBin Kim

arXiv:2608.30197v1cs.CLcs.SE

TL;DR

Existing safety steering can rely on unstable triggers and produce rigid refusals rather than constructive safe guidance. ALTSTEER combines selective intervention with staged constructive redirection, improving constructive safe responses while preserving benign utility on Llama-3.1 and Qwen2.5.

  • Problem

    Existing safety steering methods have triggers that can be unstable across domains and may remain focused on rigid refusals instead of constructive safe completions.

    Source: §1 · p. 2

  • Method

    ALTSTEER uses a refusal-relevant signal to select when to intervene, then stages steering from refusal-oriented control toward constructive alternatives during decoding.

    Source: §1 · p. 2

  • Results

    Across Llama-3.1 and Qwen2.5, ALTSTEER improves constructive safe responses to harmful requests while preserving benign utility, especially where methods tend toward short refusals.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    On the evaluated models, safe responses can move beyond short refusals while largely preserving benign utility.

    Source: §6 · p. 9

    ALTSTEER mitigates but does not eliminate safety leakage, as constructive redirection can create opportunities for harmful information to appear.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Safety alignment is essential for deploying large language models, requiring systems to prevent harmful compliance while preserving helpfulness on benign requests. Activation steering offers a training-free inference-time approach to safety control, but effective safety steering requires addressing two coupled questions: when to intervene and how generation should be shaped after intervention. However, existing safety steering methods remain limited along both dimensions, as their triggering mechanisms can be unstable across domains and refusal-oriented steering often yields rigid refusals rather than constructive safe guidance. To address these limitations, we propose ALTSTEER, an inference-time framework that couples selective intervention with refusal-anchored constructive redirection within a single inference pass. ALTSTEER uses an internal refusal-relevant signal to decide when to steer, and applies staged steering to shift generation from refusal-oriented control toward constructive alternatives. Evaluations on Llama-3.1 and Qwen2.5 show that ALTSTEER preserves benign utility while improving constructive safe-completion behavior, especially on models that otherwise tend to produce short refusals for harmful requests.

1 Introduction

ALTSTEER combines selective inference-time intervention with staged, refusal-anchored redirection toward constructive safe alternatives. Evaluations on Llama-3.1 and Qwen2.5 show improved constructive responses to harmful requests while preserving benign utility.

1 Introduction

  • Training-based safe-completion methods can be costly and inflexible, motivating inference-time activation steering that modulates safety behavior without updating model parameters (Dong et al., 2023; Turner et al., 2024; Park et al., 2024).Safe completions aim to preserve safety while offering explanations and policy-compliant alternatives (Yuan et al., 2025; Zhang et al., 2025).Source: §1 · p. 1
  • Existing steering can over-intervene on benign inputs when gating is unstable and can produce rigid refusals instead of useful safe completions on harmful requests.These limitations motivate selective intervention and refusal-anchored constructive redirection.Source: §1 · pp. 1–2
  • ALTSTEER uses an internal refusal-relevant signal to select when to steer, then gradually shifts decoding from refusal-oriented control toward constructive safe alternatives.The schedule emphasizes refusal anchoring early in decoding and constructive alternatives later.Source: §1 · p. 2, Figure 1 · p. 2
  • Across Llama-3.1 and Qwen2.5, ALTSTEER improves constructive safe responses to harmful requests while preserving benign utility within a single inference pass.The evaluation covers Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct.Source: §1 · p. 2

2 Related Work

Prior work moves safety beyond bare refusals toward informative safe completions and selectively applies refusal steering, while ALTSTEER preserves the refusal boundary and redirects responses constructively during decoding.

2.1 Constructive Safe Completions

  • Post-training work promotes explanatory safe responses and alternatives beyond brittle, surface-level refusals, while ALTSTEER preserves the refusal boundary as it shapes constructive completions.SAFESTEER instead steers toward category-specific safe-content directions at inference time.Source: §2.1 · p. 3

2.2 Activation Steering

  • Selective activation-steering methods decide when or how strongly to apply refusal control while aiming to limit interference with benign behavior.CAST uses conditional activation triggers (Lee et al., 2025), AdaSteer adjusts steering strength using harmfulness and rejection directions (Zhao et al., 2025b), SafeSwitch uses a safety prober (Han et al., 2025), and AlphaSteer learns input-dependent steering under a null-space constraint (Sheng et al., 2026).Source: §2.2 · p. 3
  • Deciding when to intervene does not determine how responses should be shaped; ALTSTEER coordinates refusal and alternative steering during decoding for refusal-anchored constructive redirection.Source: §2.2 · p. 3

3 Empirical Analysis

The section tests whether existing safety-steering gates generalize across domains and whether intervention produces more than rigid, template-based refusals.

3.1 Instability of Existing Gating Signals Across Domains

  • CAST (Lee et al., 2025) over-triggers on benign math, Latent Guard (Zhao et al., 2025a) misses adversarial harms, while ALTSTEER yields a more consistent cross-domain boundary.The comparison evaluates benign MATH500 and GSM8K queries against harmful AdvBench and JailbreakBench prompts.Source: §3.1 · pp. 3–4

3.2 Template-based Responses in Existing Methods

  • Across harmful benchmarks, existing steering methods often produce short, fixed template refusals; even SAFESTEER retains a substantial proportion.Template refusals are defined as short, fixed responses with little or no explanatory or constructive continuation.Source: §3.2 · p. 4, Figure 3 · p. 4
  • Effective safety steering must shape safe-response form during generation, moving beyond hard rejection toward constructive redirection.Deciding when to intervene alone is insufficient.Source: §3.2 · p. 4

4 Method

ALTSTEER couples a refusal-relevant signal for selective intervention with staged steering that shifts generation from refusal-oriented control toward constructive alternatives in one inference pass. It uses separate refusal and alternative directions to shape that trajectory.

4.1 Overview

  • ALTSTEER modifies hidden states with layer-wise refusal and orthogonalized alternative vectors, using step-dependent steering strengths during decoding.Its staged trajectory begins with refusal-oriented control and gradually shifts toward constructive alternatives while maintaining a refusal anchor.Source: §4.1 · p. 4

4.2 Refusal-relevant Internal Signals for Selective Intervention

  • ALTSTEER gates intervention by average cosine alignment between final-token hidden states and layer-wise refusal vectors, steering only when the score is positive.The zero boundary is parameter-free; averaging across layers is intended to reduce sensitivity to noise at any one layer.Source: §4.2 · pp. 4–5

4.3 Refusal Vector

  • The refusal vector is a layer-wise difference-in-means contrast between harmful prompts the naïve model refuses and harmful prompts it complies with.Using harmful prompts in both sets is intended to isolate refusal-related behavior while reducing features associated with harmful input content.Source: §4.3 · p. 5

4.4 Alternative Vector and Orthogonalization

  • The alternative vector contrasts explained and simple refusals elicited from the same harmful inputs; its refusal-direction projection is removed to emphasize constructive continuation.The response styles are elicited with different prompts, and the construction is examined for sensitivity to paraphrased explained-refusal prompts.Source: §4.4 · pp. 5–6

4.5 Staged Steering

  • ALTSTEER progressively shifts steering strength from the refusal vector toward the orthogonalized alternative vector as decoding proceeds.This schedule is designed to start with refusal-oriented behavior and transition toward explanatory or constructive alternatives.Source: §4.5 · p. 6

5 Experiments

ALTSTEER improves constructive safe responses to harmful requests, especially on Llama-3.1, while preserving benign utility and remaining competitive on Qwen2.5. Ablations show that selective gating and staged refusal-anchored steering support this balance.

5.1 Experimental Setup

  • Experiments use Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct across harmful-request, over-safety, instruction-following, and math benchmarks.Baselines include CAST, AdaSteer, SafeSwitch, and SAFESTEER.Source: §5.1 · p. 6

5.2 Experimental Results

  • ALTSTEER raises Alternative Rate and Refusal-and-Alternative Rate (RAR), especially on Llama-3.1, and achieves the best RAR on AdvBench and MaliciousInstruct.On Qwen2.5, which already shows stronger safe-completion behavior, the gap between methods narrows, but ALTSTEER remains competitive.Source: Table 2 · p. 7, §5.2 · p. 7
  • On Llama-3.1, Safe Alt. rose from 22.4% (base model) to 61.1% (ALTSTEER); on Qwen2.5, ALTSTEER reached 83.1%, below SafeSwitch’s 90.5%.ALTSTEER’s increase on Llama-3.1 came with a higher leakage rate, but it still produced the largest overall proportion of safe alternatives.Source: Table 3 · p. 7, §5.2 · pp. 7–8
  • ALTSTEER avoids severe utility degradation and stays close to the base model across benign benchmarks, unlike prior steering methods that can degrade mathematical reasoning performance.SafeSwitch also shows that strong refusal control on harmful requests can degrade XSTest compliance on benign safety-sensitive prompts.Source: Table 4 · p. 8, §5.2 · p. 8

5.3 Ablation Study and Analysis

  • All-layer gating yields the highest precision across both models, whereas last-layer gating leads in overall accuracy and F1; single-layer gating is brittle across architectures.Because false activations can harm benign utility, the authors adopt all-layer gating as a conservative default.Source: Table 5 · p. 8, §5.3 · pp. 8–9
  • Refusal-only steering minimizes leakage but rarely provides alternatives, while alternative-only steering increases alternatives at the cost of the highest leakage; ALTSTEER achieves the best trade-off.The early refusal anchor constrains generation as alternative steering strengthens, supporting constructive redirection with less leakage than unanchored alternative-only steering.Source: Table 6 · p. 8, §5.3 · p. 9
  • Increasing λref tends to raise Refusal Rate, while Alternative Rate responds more to λalt; excessive λalt can weaken refusal or alternative quality.The selected balanced configuration is λref = 1.5 and λalt = 1.0.Source: Table 7 · p. 8, §5.3 · p. 9

6 Conclusion

ALTSTEER combines refusal-relevant gating with staged constructive redirection at inference time, guiding responses from refusal-oriented control toward constructive alternatives while preserving the refusal boundary. On Llama-3.1 and Qwen2.5, it improves refusal-anchored constructive redirection while largely preserving benign utility, especially where existing methods tend toward short refusals.

6 Conclusion

  • ALTSTEER dynamically guides responses from refusal-oriented control toward constructive alternatives while preserving the refusal boundary.The method is presented as a staged activation steering approach.Source: §6 · p. 9
  • ALTSTEER improves refusal-anchored constructive redirection while largely preserving benign utility on Llama-3.1 and Qwen2.5, especially where existing methods yield short refusals.The framework operates within a single inference pass.Source: §6 · p. 9
  • At inference time, ALTSTEER combines refusal-relevant gating with staged constructive redirection.The gating signal determines when to steer, while staged steering shifts generation toward constructive alternatives.Source: §6 · p. 9

Limitations … Utility Evaluation

ALTSTEER’s limitations include dependence on reference-vector quality, residual safety leakage, and a single-pass trade-off against post-hoc rewriting. The accompanying sections describe how its steering vectors are constructed and list harmful, over-safety, and utility evaluation datasets.

Limitations

  • ALTSTEER’s alternative vector may offer limited support when constructive safe-completion behavior is weakly represented in the base model.The vector is extracted from activations elicited by response-style prompts, and more systematic vector construction remains future work.Source: Limitations · p. 9
  • ALTSTEER mitigates but does not eliminate safety leakage, while post-hoc rewriting may improve absolute response quality at the cost of extra generation passes.ALTSTEER is framed as a complementary single-pass, white-box steering trade-off rather than a universally superior alternative.Source: Limitations · p. 9

A.1 Dataset used to create refusal vector and alternative vector

  • From 2,500 randomly sampled BeaverTails training examples, the method estimates contrastive activation statistics for its refusal and alternative vectors.The samples are drawn from the training split.Source: §A.1 · p. 12
  • The refusal vector contrasts mean hidden activations for harmful prompts the naïve model refuses versus those it complies with.For the alternative vector, explained refusals are positive examples and simple refusals are negative examples; their mean-activation difference defines the direction.Source: §A.1 · p. 12

Harmful Dataset

  • Harmful evaluation uses 625 sampled unsafe BeaverTails test examples, alongside AdvBench’s 520 adversarial instructions and 100 prompts each from MaliciousInstruct and JailbreakBench.BeaverTails covers 14 harm categories; MaliciousInstruct spans risk categories, and JailbreakBench contributes harmful behaviors.Source: Harmful Dataset · p. 12

Over-Safety Evaluation

  • Over-safety evaluation uses XSTest’s 250 safe prompts across ten categories that well-calibrated models should readily answer.The benchmark tests whether models comply with safe requests.Source: Over-Safety Evaluation · p. 12

Utility Evaluation

  • Utility evaluation includes AlpacaEval’s broad question-answering benchmark, MATH500’s 500 competition problems, and GSM8K’s 8,500 multistep grade-school math problems.AlpacaEval compares model answers with reference answers using an automated system.Source: Utility Evaluation · p. 12

B Baseline Methods · C Evaluation Details · C.1 Evaluation Prompts

The section identifies four recent safety-steering baselines and specifies the prompts and scoring procedures used to evaluate ALTSTEER’s safety and benign-task behavior.

B Baseline Methods

C.1 Evaluation Prompts

  • RAR uses the Table 8 prompt, XSTest compliance follows Röttger et al. (2024) with the Table 9 prompt, and GPT-4o at temperature 0 evaluates both.Source: §C.1 · p. 13
  • Leakage Rate on alternatives uses the Table 10 prompt; MATH500 and GSM8K accuracy compares extracted final numerical answers with benchmark ground truth.Source: §C.1 · p. 13

C.2 Human and Cross-Judge Validation · D Additional Analysis of Refusal-Relevant Gating · D.1 Threshold Choice for Refusal-Relevant Gating

Human and cross-judge checks support the reliability of the automatic evaluations, while a threshold analysis supports zero as a parameter-free refusal-gating boundary that performs best among tested thresholds on both models. Under GPT-4o-mini, ALTSTEER retains the highest Safe Alternative Rate among the compared methods.

C.2 Human and Cross-Judge Validation

  • The validation study sampled 100 harmful prompts and collected 300 Llama-3.1 responses from the naïve model, CAST, and ALTSTEER for human and GPT-4o labeling.Three human annotators and GPT-4o judged explicit refusal and constructive-alternative presence for each response.Source: §C.2 · p. 13
  • GPT-4o agrees with the human majority on 99.0% of Refusal and 91.3% of Alternative judgments.Alternative judgments are less aligned, consistent with constructive alternatives being more subjective to identify than explicit refusals.Source: §C.2 · p. 13
  • On 200 alternative-providing responses from AdaSteer and ALTSTEER, GPT-4o matched the human majority on 98.0% of leakage labels (Cohen’s κ = 0.943).All four disagreements involved GPT-4o flagging leakage when the human majority did not, suggesting slightly conservative automatic labels in this sample.Source: §C.2 · p. 13
  • 51.9% Safe Alternative Rate under GPT-4o-mini keeps ALTSTEER ahead of AdaSteer at 29.9%.The main method-level conclusion is unchanged despite GPT-4o-mini flagging more leakage than GPT-4o.Source: §C.2 · p. 13

D.1 Threshold Choice for Refusal-Relevant Gating

  • Zero is the parameter-free sign boundary for the refusal-relevant gate: positive refusal-alignment scores trigger steering, while non-positive scores leave the model unchanged.The score averages cosine alignment between the input representation and refusal direction, rather than using a cutoff tuned through validation-set search.Source: §D.1 · p. 13
  • Zero achieved the best overall accuracy and F1 among tested thresholds on Llama-3.1-8B and Qwen2.5-7B while maintaining high precision.Lower thresholds broaden steering and reduce precision and accuracy; higher thresholds reduce recall substantially.Source: §D.1 · p. 14
  • The analysis supports zero as a parameter-free accuracy–F1 trade-off among tested thresholds, not as a universally optimal classifier threshold.This choice follows from the sign-based definition of the refusal-alignment signal.Source: §D.1 · p. 14

D.2 Robustness under Multi-Turn Context · E Prompt Sensitivity of the Alternative Vector · F Analysis of Alternative Steering Schedules

ALTSTEER’s refusal-relevant gate remains largely stable in a minimal two-turn context, while alternative-vector prompt variants improve harmful-response refinement but can differ in redirection strength.

D.2 Robustness under Multi-Turn Context

  • The multi-turn gate test places a benign, topically related question and model response before each harmful request from MaliciousInstruct and JailbreakBench.The refusal-relevant gate score is computed from the final-token representation of the full context.Source: §D.2 · p. 14
  • Gate activation decreases only slightly, from 93.0% in single-turn prompts to 89.5% under multi-turn context.The refusal-alignment signal remains largely stable in this minimal two-turn context-accumulation setting.Source: §D.2 · p. 14

E Prompt Sensitivity of the Alternative Vector

  • Alternative vectors were constructed with three prompt-pair variants, each contrasting an explained refusal with a simple refusal for the same harmful inputs.This tests whether the vector depends on the exact wording of the extraction prompt.Source: E Prompt Sensitivity of the… · p. 14
  • All three extraction-prompt variants substantially improve RAR over base Llama-3.1 on MaliciousInstruct.The results indicate that ALTSTEER is not tied to a single brittle prompt template.Source: E Prompt Sensitivity of the… · p. 14
  • Prompt wording can still affect the strength of constructive redirection, despite improvements across all tested variants.The variants contrast explained and simple refusals for the same harmful inputs.Source: E Prompt Sensitivity of the… · p. 14

F.1 Comparison of Scheduling Strategies · F.2 Internal Dynamics of Staged Steering · G Comparison with Two-Stage Post-Refusal Prompting

The comparisons show that ALTSTEER’s staged schedule supports constructive redirection, while its single-pass approach trades off against a two-stage baseline that achieves higher RAR on several harmful benchmarks but requires additional model calls. Internal refusal alignment shifts from early refusal-oriented generation toward later constructive continuations, a pattern consistent with the staged behavior but not causal proof of its mechanism.

F.1 Comparison of Scheduling Strategies

  • The schedule comparisons test constant-strength mixing, reversed refusal-to-alternative strengths, a first-sentence hard switch, and response-aware control based on refusal alignment.Constant mixture uses the staged schedule’s average strengths; response-aware steering decreases refusal strength and increases alternative strength when refusal alignment strengthens.Source: §F.1 · pp. 15–16

F.2 Internal Dynamics of Staged Steering

  • The internal-dynamics analysis tracks pre-steering refusal alignment at Llama-3.1 layer 13 across decoding on 200 harmful prompts from MaliciousInstruct and JailbreakBench.Because the hidden state is measured before steering at each step, the trajectory reflects effects of earlier steering on subsequent generation.Source: §F.2 · p. 16
  • Refusal alignment starts around 0.08–0.10, crosses zero at about step 25, and settles near −0.05 as constructive continuations emerge; this is not causal proof.The authors interpret the trajectory as an internal representational transition associated with the staged schedule, not causal evidence of the mechanism.Source: §F.2 · p. 16

G Comparison with Two-Stage Post-Refusal Prompting

  • The two-stage baseline detects short refusals lacking constructive cues and triggers a rewrite prompt only for those responses.Its rule-based detector checks common refusal markers and excludes constructive cues; the second-stage prompt asks for a more helpful refusal with a related safe guideline.Source: G Comparison with Two-Stage… · pp. 16–17

F.1 Comparison of Scheduling Strategies

  • Constant, Reverse, and Hard switch reduce RAR versus staged steering, with Hard switch nearly eliminating constructive alternatives; Response-aware slightly outperforms staged steering.The comparison uses MaliciousInstruct and JailbreakBench on Llama-3.1, with selective gating disabled to isolate scheduling effects.Source: §F.1 · p. 16, Table 16 · p. 17

G Comparison with Two-Stage Post-Refusal Prompting

  • Two-stage prompting achieves higher RAR than ALTSTEER on several harmful benchmarks but requires 1.614 model calls per input on average, versus ALTSTEER’s single inference pass.The comparison reports RAR on harmful benchmarks and average model calls per input.Source: Table 18 · p. 17, G Comparison with Two-Stage… · p. 17
  • ALTSTEER steers internal activations in one pass to preserve the refusal boundary while encouraging constructive redirection, unlike limited-response-form steering baselines and post-hoc prompting.Two-stage prompting refines detected bare refusals through an additional detector-and-rewrite step.Source: G Comparison with Two-Stage… · p. 17

H Additional Model Transfer Results · I Qualitative Case Study

Preliminary transfer checks find that ALTSTEER improves safe-completion behavior on both additional models, but its utility trade-offs vary with the base model’s alignment profile. A qualitative case study shows that refusal anchoring enables constructive redirection without the unsafe leakage seen with alternative-only steering.

H Additional Model Transfer Results

  • The additional evaluations are preliminary transfer checks across model scale and alignment profiles, using Qwen2.5-14B-Instruct and Mistral-7B-Instruct-v0.3.Source: H Additional Model Transfer Results · p. 18
  • On Qwen2.5-14B-Instruct, ALTSTEER improves RAR on JailbreakBench and MaliciousInstruct despite limited headroom, while broadly preserving utility with moderate decreases on XSTest and MATH500.The model already exhibits strong safe-completion behavior.Source: H Additional Model Transfer Results · p. 18
  • ALTSTEER’s transfer benefits vary by model: it improves harmful-response quality on both models, but utility degradation is more noticeable for Mistral-7B-Instruct-v0.3.Qwen2.5-14B-Instruct broadly preserves utility with moderate decreases on XSTest and MATH500; Mistral shows noticeable utility degradation on MATH-500 and XSTest.Source: H Additional Model Transfer Results · p. 18
  • On Mistral-7B-Instruct-v0.3, ALTSTEER most clearly improves harmful-response quality on JailbreakBench, with marginal gains on MaliciousInstruct and noticeable utility degradation on MATH-500 and XSTest.The results suggest transfer beyond one model family, while effectiveness varies with the base model’s alignment profile and latent safe-completion behavior.Source: H Additional Model Transfer Results · p. 18

I Qualitative Case Study

  • Prior refusal-oriented methods stay brief and rigid, whereas Alternative-only steering provides explanatory content but leaks unsafe details, including a toy SQL injection script.Table 20’s case study concerns a harmful request on Llama-3.1.Source: I Qualitative Case Study · p. 18, Table 20 · p. 19
  • ALTSTEER maintains a firm refusal boundary while redirecting the harmful request toward safe guidance, including responsible vulnerability identification and testing.This contrasts with both rigid refusals and unsafe alternative-only explanatory content.Source: I Qualitative Case Study · p. 18, Table 20 · p. 19
  • The case study supports the finding that steering for explanation without a safety anchor can inadvertently lower the safety bar.Source: I Qualitative Case Study · p. 18

Related papers

Loading 2608.30197v1…