PaperlayerView exact revision on arXiv

Source-linked AI summary

ALTSTEER: Selective Safety Steering for Moving Beyond Hard Refusals to Constructive Alternatives

Hoejoon Kwon, Byeonggeuk Lim, Kahyeon Kim, YoungBin Kim

arXiv:2608.30197v1cs.CLcs.SE

TL;DR

Existing safety steering can use unstable intervention triggers and produce rigid refusals rather than constructive safe guidance. ALTSTEER combines refusal-relevant selective gating with staged redirection in one inference pass, improving constructive safe responses while preserving benign utility.

  • Problem

    Existing safety steering is limited by unstable cross-domain triggering and refusal-oriented outputs that often remain rigid rather than constructively safe.

  • Method

    ALTSTEER uses an internal refusal-relevant signal for selective intervention and staged steering that shifts generation from refusal control toward constructive alternatives.

  • Results

    ALTSTEER improves constructive safe responses to harmful inputs while preserving benign utility on Llama-3.1 and Qwen2.5.

  • Takeaways & Limitations

    Inference-time safety steering can combine a firm refusal boundary with explanatory and constructive safe completions in a single pass.

  • Takeaways & Limitations

    ALTSTEER depends on reference-vector quality, and constructive redirection can still create opportunities for harmful information to appear.

Abstract

from arXiv · show

Safety alignment is essential for deploying large language models, requiring systems to prevent harmful compliance while preserving helpfulness on benign requests. Activation steering offers a training-free inference-time approach to safety control, but effective safety steering requires addressing two coupled questions: when to intervene and how generation should be shaped after intervention. However, existing safety steering methods remain limited along both dimensions, as their triggering mechanisms can be unstable across domains and refusal-oriented steering often yields rigid refusals rather than constructive safe guidance. To address these limitations, we propose ALTSTEER, an inference-time framework that couples selective intervention with refusal-anchored constructive redirection within a single inference pass. ALTSTEER uses an internal refusal-relevant signal to decide when to steer, and applies staged steering to shift generation from refusal-oriented control toward constructive alternatives. Evaluations on Llama-3.1 and Qwen2.5 show that ALTSTEER preserves benign utility while improving constructive safe-completion behavior, especially on models that otherwise tend to produce short refusals for harmful requests.

1 Introduction

ALTSTEER addresses unstable intervention triggers and rigid refusal outputs by combining selective inference-time steering with staged redirection toward constructive safe alternatives. Evaluations report improved constructive responses to harmful inputs while preserving benign utility.

  • Existing safety alignment often ends harmful requests with short, rigid refusals, reducing utility compared with explanatory or constructive safe responses.
  • When activated, steering begins with refusal-oriented control and gradually shifts generation toward constructive safe alternatives during decoding.
  • Its internal refusal-relevant signal determines whether steering is applied, targeting unnecessary intervention on benign inputs.
  • The introduction identifies unstable selective triggers and rigid refusal-oriented steering as the two limitations ALTSTEER targets.
  • ALTSTEER couples selective intervention with staged, refusal-anchored constructive redirection in a single inference pass.
  • ALTSTEER produces more constructive safe responses to harmful inputs while preserving utility on benign inputs across Llama-3.1 and Qwen2.5.

2 Related Work

Related work frames safety steering as inference-time control over latent behavioral directions, while highlighting a gap between deciding when to intervene and shaping responses afterward. ALTSTEER targets that gap with refusal-anchored constructive redirection.

  • Safety research has shifted from refusal-centered prevention toward informative safe responses with explanations, rationales, and policy-compliant alternatives.
  • Activation steering injects latent direction vectors into hidden activations to modulate behaviors without updating model parameters.
  • Existing selective-control approaches address intervention timing, but deciding when to intervene does not determine how responses should be shaped afterward.
  • ALTSTEER additionally coordinates refusal and alternative steering during decoding to target refusal-anchored constructive redirection.

3 Empirical Analysis

The empirical analysis examines whether gating signals remain stable across domains and whether steering produces responses beyond template refusals. It finds more consistent intervention behavior for ALTSTEER and persistent template responses for existing methods.

  • The analysis asks whether existing gating signals provide stable intervention boundaries across heterogeneous datasets.
  • It also tests whether steering methods shape safe responses beyond rigid, template-based refusals.
  • The comparison evaluates CAST, Latent Guard, and ALTSTEER’s refusal-direction-aligned gating signal on benign and harmful domains.
  • ALTSTEER yields a more consistent intervention boundary, while CAST produces benign mathematical false positives and Latent Guard misses many adversarial harmful prompts.
  • Template Refusal Rate measures the proportion of short, fixed refusal responses with little or no explanatory or constructive continuation.
  • Existing steering methods frequently remain standardized and repetitive across harmful benchmarks, including SAFESTEER despite steering toward non-refusal safe content.
  • Effective safety steering therefore requires shaping safe-response form during generation, not only deciding whether intervention is triggered.

4 Method

ALTSTEER combines selective intervention with refusal-anchored constructive redirection in one inference pass. It detects refusal-relevant alignment across layers, constructs complementary refusal and alternative directions, and stages their influence during decoding.

  • Overview: ALTSTEER extends activation steering by coupling selective intervention with refusal-anchored constructive redirection within a single inference pass.The framework modifies hidden activations using refusal and orthogonalized alternative steering vectors.
  • Refusal-relevant Internal Signals for Selective Intervention: The intervention signal averages layer-wise cosine similarity between each input’s final-token hidden state and the corresponding refusal vector.Positive average alignment indicates refusal-relevant behavior and provides a parameter-free zero boundary for intervention.
  • Refusal Vector: The refusal vector contrasts harmful prompts the naïve model refuses with harmful prompts it complies with, isolating refusal-related behavior from harmful-input content.Mean hidden states are extracted from the final token position at each layer before computing the contrastive direction.
  • Alternative Vector and Orthogonalization: The alternative vector contrasts explained and simple refusals elicited from the same harmful inputs using different response-style system prompts.This construction targets constructive continuation following a refusal anchor, and its sensitivity to paraphrased prompts is examined separately.
  • Alternative Vector and Orthogonalization: ALTSTEER removes the alternative vector’s projection onto the refusal vector to emphasize constructive continuation while reducing overlap with refusal behavior.The resulting orthogonalized alternative vector complements the initial refusal anchor.
  • Staged Steering: Steering strengths are scheduled across decoding steps so generation begins with refusal-oriented control and progressively shifts toward explanatory or constructive alternatives.The refusal and alternative vectors therefore play complementary roles during decoding.

5 Experiments

Experiments evaluate ALTSTEER on harmful and benign benchmarks using two safety-aligned models, showing stronger constructive safe responses and preserved utility. Ablations further examine alternative safety metrics, gating choices, and staged steering.

  • Experimental Setup: Experiments use Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct across harmful, utility, instruction-following, and mathematical-reasoning benchmarks.Transfer results on Qwen2.5-14B-Instruct and Mistral-7B-Instruct-v0.3 are reported in Appendix H.
  • Experimental Setup: Harmful-benchmark evaluation reports Refusal Rate, Alternative Rate, Refusal-and-Alternative Rate, Leakage Rate, and Safe Alt.RAR measures responses that both refuse harmful requests and provide alternatives; Safe Alt. measures the proportion providing a safe alternative.
  • Constructive Safe Responses: ALTSTEER increases Alternative Rate and RAR, especially on Llama-3.1, where refusal-oriented baselines often produce strong refusals but limited constructive alternatives.On Qwen2.5, the performance gap is smaller, but ALTSTEER achieves the best RAR on AdvBench and MaliciousInstruct.
  • Safety of Constructive Alternatives: 61.1% Safe Alt. for ALTSTEER on Llama-3.1 exceeds the base model’s 22.4%, while Qwen2.5 results remain competitive at 83.1%.On Llama-3.1, the increase is accompanied by a higher leakage rate; on Qwen2.5, SafeSwitch reaches 90.5%.
  • Utility Preservation: ALTSTEER remains close to the base model across benign benchmarks and avoids the severe utility degradation observed with prior steering methods.SafeSwitch illustrates that strong harmful-side refusal control can still degrade XSTest utility, whereas refusal-relevant gating reduces unnecessary intervention on benign inputs.
  • Gating Robustness: All-layer gating provides the highest precision across both models, while single-layer gating is brittle across architectures; ALTSTEER therefore adopts all-layer gating.Last-layer gating has the highest overall accuracy and F1, but all-layer gating better prioritizes avoiding false positives on benign inputs.
  • Ablation Study: ALTSTEER’s refusal anchor achieves the best trade-off between safe alternatives and leakage compared with refusal-only and alternative-only steering.The early refusal anchor constrains generation as the alternative direction strengthens, supporting staged steering as a balance between safety anchoring and helpful redirection.
  • Steering Strengths: Increasing λref tends to raise Refusal Rate, whereas Alternative Rate is more sensitive to λalt and moderate settings yield stronger constructive redirection.Further increasing λalt beyond the moderate range does not consistently improve performance.

6 Conclusion

ALTSTEER combines refusal-relevant gating with staged constructive redirection during inference. Experiments on Llama-3.1 and Qwen2.5 report improved refusal-anchored constructive responses while largely preserving benign utility.

  • Conclusion: ALTSTEER dynamically guides responses from refusal-oriented control toward constructive alternatives while preserving the refusal boundary.The method combines refusal-relevant gating with staged constructive redirection at inference time.
  • Conclusion: Experiments on Llama-3.1 and Qwen2.5 show improved refusal-anchored constructive redirection with largely preserved benign utility.The reported gains are especially notable where existing methods remain dominated by short refusals.

Limitations

ALTSTEER’s limitations concern reference-vector quality, residual safety leakage, and its single-pass operating point relative to post-hoc rewriting.

  • Limitations: ALTSTEER depends on the quality of its reference vectors, especially the orthogonalized alternative vector.The vector is extracted from activations elicited by specific response-style prompts.
  • Limitations: Weakly represented constructive and explanatory behavior may limit the extracted direction’s support for meaningful alternatives.More systematic vector construction remains future work.
  • Limitations: ALTSTEER mitigates but does not eliminate safety leakage during constructive redirection.Further reducing leakage while preserving constructive alternatives remains future work.
  • Limitations: Post-hoc rewriting may achieve stronger absolute response quality when additional generation passes are acceptable.ALTSTEER instead targets a single-pass white-box steering operating point.

A.2 Benchmarks Harmful Dataset

The evaluation combines harmful-safety benchmarks with benign utility, instruction-following, and mathematical-reasoning datasets.

  • Harmful datasets: BeaverTails contains 330k labeled user prompts and LLM responses spanning 14 harm categories.The evaluation samples 625 unsafe test examples.
  • Harmful datasets: AdvBench contains 520 adversarial harmful instructions for testing resistance to unsafe behavior under attack-style prompts.
  • Harmful datasets: MaliciousInstruct contains 100 malicious prompts spanning categories including crime, violence, and harmful activities.
  • Harmful datasets: JailbreakBench’s harmful subset contains 100 harmful behaviors for evaluating jailbreak robustness.
  • Utility and general ability: XSTest provides 250 safe prompts across ten categories for testing over-safety, while AlpacaEval assesses broad question answering.
  • Utility and general ability: MATH500 contains 500 high-level competition problems, and GSM8K contains 8,500 multi-step grade-school math problems.

B Baseline Methods

ALTSTEER is compared with inference-time safety steering baselines covering selective refusal control and steering toward non-refusal safe content.

  • Baseline methods: The evaluation compares ALTSTEER with CAST, AdaSteer, SafeSwitch, and SAFESTEER as recent safety steering methods.
  • Baseline methods: CAST uses conditional vectors to classify prompts and selectively intervene when inputs are judged harmful.
  • Baseline methods: AdaSteer adaptively changes steering strength using an internal harmfulness signal to increase refusal control for harmful intent.It reduces intervention for benign inputs.
  • Baseline methods: SafeSwitch uses a trained safety prober to activate selective refusal-oriented intervention when unsafe behavior is predicted.
  • Baseline methods: SAFESTEER biases generation toward safe-content directions in activation space without explicitly retraining the model.

C.2 Human and Cross-Judge Validation

Human and cross-judge checks validate the automatic refusal, alternative, and leakage evaluations, while threshold analysis examines the refusal-relevant gate.

  • Human validation: 300 Llama-3.1 responses were independently labeled by three human annotators and GPT-4o for refusal and constructive alternatives.The sample covers 100 harmful prompts from four benchmarks and three model conditions.
  • Human validation: GPT-4o achieved 99.0% agreement with human majority votes for Refusal and 91.3% for Alternative.Alternative judgments were less aligned because constructive explanations and alternatives are more subjective.
  • Leakage validation: GPT-4o agreed with human majority leakage labels on 98.0% of 200 alternative-providing responses.The validation sampled equally from AdaSteer and ALTSTEER.
  • Cross-judge validation: GPT-4o and GPT-4o-mini agreed on 93.1% of RAR labels and 90.3% of leakage labels.GPT-4o-mini flagged 15.8% leakage versus 7.7% for GPT-4o, while the main method-level conclusion remained unchanged.
  • Threshold choice: Zero is a parameter-free sign-based intervention boundary for the refusal-relevant gate.Positive refusal alignment triggers steering, whereas non-positive scores leave the model unchanged.
  • Threshold choice: The zero threshold achieved the best overall accuracy and F1 among tested thresholds on both Llama-3.1-8B and Qwen2.5-7B.Lower thresholds reduce precision and accuracy, while higher thresholds substantially reduce recall.

D.2 Robustness under Multi-Turn Context

ALTSTEER remains robust across accumulated context, alternative-vector extraction prompts, and steering schedules. Its staged, single-pass design supports constructive redirection while preserving a distinct deployment trade-off against two-stage prompting.

  • Robustness under Multi-Turn Context: 89.5% gate activation under two-turn context, versus 93.0% in single-turn inputs, indicates largely stable refusal-relevant gating.The evaluation uses a benign related question and response before each harmful request.
  • Alternative-Vector Prompt Robustness: All tested prompt-pair variants substantially improve RAR over base Llama-3.1, although wording still affects redirection strength.The variants contrast explained and simple refusals for the same harmful inputs.
  • Scheduling Ablation: Constant, Reverse, and Hard switch schedules reduce RAR relative to staged steering, while Response-aware slightly outperforms it.These comparisons hold steering vectors, intervention layer, and decoding settings fixed; Hard switch nearly eliminates constructive alternatives.
  • Internal Trajectory: Refusal alignment starts near 0.08–0.10, crosses zero around step 25, and settles near −0.05 as constructive continuations emerge.The trajectory matches the intended transition from refusal-oriented openings to safer explanatory redirection.
  • Deployment Trade-off: Two-stage prompting achieves higher RAR on several harmful benchmarks but averages 1.614 model calls per input, whereas ALTSTEER operates in one inference pass.The baseline uses a response-level bare-refusal detector followed by a rewrite generation call.

H Additional Model Transfer Results

Transfer checks on Qwen2.5-14B-Instruct and Mistral-7B-Instruct-v0.3 show that ALTSTEER improves harmful-response quality beyond the primary models, but utility and gains vary with the base model’s alignment profile.

  • Qwen2.5-14B-Instruct: ALTSTEER improves RAR on both JailbreakBench and MaliciousInstruct for Qwen2.5-14B despite limited headroom.Utility is broadly preserved, with moderate decreases on XSTest and MATH500.
  • Mistral-7B-Instruct-v0.3: On Mistral-7B-Instruct-v0.3, harmful-response gains are clearest on JailbreakBench but marginal on MaliciousInstruct.The model also shows noticeable utility degradation on MATH-500 and XSTest.
  • Qualitative Case Study: The case study shows ALTSTEER combining a firm refusal boundary with safe guidance, unlike alternative-only steering that leaks unsafe details.The comparison contrasts rigid refusals, explanatory leakage, and constructive redirection.
Loading 2608.30197v1…