AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraint

1 INTRODUCTION

• Directly applying refusal vectors creates over-refusal and degraded benign-task performance, motivating a principled solution to the safety–utility trade-off.Existing calibration and conditional-steering approaches are described as largely heuristic and lacking theoretical grounding.Source: §1 · p. 2
• The method constrains its learned transformation so benign prompts receive nearly zero steering, preserving their activations while malicious prompts are mapped toward refusal.It uses s = ∆h, with ∆ constrained to the null space of benign activations and trained to satisfy ∆h_m ≈ r.Source: §1 · p. 3
• AlphaSteer consistently outperforms activation-steering baselines across jailbreak attacks, achieving an average DSR above 90%.The authors attribute this to learned reconstruction of refusal directions for malicious activations.Source: §1 · p. 3
• AlphaSteer maintains utility close to vanilla-model performance, whereas baseline steering methods show unstable degradation on general capabilities.The introduction also reports that benign activations remain largely unchanged as steering strength increases.Source: §1 · p. 3

2.1 INDUCING REFUSAL VIA ACTIVATION STEERING

• Activation steering injects a refusal direction vector into selected-layer activations, shifting model behavior from compliance toward refusal.The refusal vector is commonly computed as the mean difference between refusal and compliance activations.Source: §2.1 · p. 3
• The refusal direction is formed from mean activations of refusal and compliance prompt sets collected at the last token position.These sets are denoted D_r and D_c in the difference-in-means construction.Source: §2.1 · p. 3

2.2 LITERATURE REVIEW

• Indiscriminate refusal-vector injection can over-refuse benign prompts, creating a trade-off between safety enhancement and utility preservation.This trade-off makes direct activation steering difficult to deploy in real-world settings.Source: §2.2 · pp. 3–4
• Vector calibration modifies the refusal direction for better malicious-prompt targeting, while conditional steering activates it only when prompts are predicted as malicious.The latter adjusts the steering strength λ using activation-based conditions or thresholds.Source: §2.2 · p. 4
• These approaches rely heavily on empirical calibration rules or manually crafted conditions, raising concerns about robustness and generalizability.The stated limitation motivates theoretically grounded methods that preserve benign utility while inducing malicious-prompt refusal.Source: §2.2 · p. 4

3.1 LEARNABLE ACTIVATION STEERING FOR PRINCIPLED CONTROL

• AlphaSteer dynamically constructs a prompt-dependent steering vector s(l) = ∆(l)h(l) using a learnable transformation matrix rather than fixed vectors and strengths.The transformation matrix is optimized for utility preservation and safety enhancement.Source: §3.1 · p. 5

3.2 UTILITY PRESERVATION WITH NULL SPACE PROJECTION

• For benign prompts, AlphaSteer constrains the steering transformation to the null space of benign activations so the steering term vanishes.The benign activation matrix Hb contains sampled prompt activations, and the constraint is ∆Hb = 0.Source: §3.2 · pp. 5–6
• AlphaSteer computes the null-space projection efficiently from the non-central covariance matrix HbH⊤, exploiting null-space equivalence when the activation dimension is much smaller than the number of data points.SVD retains eigenvectors with zero eigenvalues to span the null space, forming the projection matrix.Source: §3.2 · pp. 5–6

3.3 SAFETY ENHANCEMENT WITH REFUSAL DIRECTION VECTOR RECONSTRUCTION

• For malicious prompts, AlphaSteer reconstructs a refusal direction by fitting a transformation from malicious activations to repeated refusal vectors with regularized least squares.The regularization uses the Frobenius norm to avoid overfitting, and the solution reconstructs a refusal vector for steering malicious activations.Source: §3.3 · p. 6

3.4 ALPHASTEER

• The final steering function combines the learned transformation with the null-space projection, steering malicious activations toward refusal while leaving benign activations largely unchanged.This design targets safety enhancement without compromising general capabilities.Source: §3.4 · p. 6, §3.2 · p. 6

4.1 SAFETY ENHANCEMENT (RQ1)

• AlphaSteer consistently outperforms activation-steering baselines across selected jailbreak attacks, achieving an average DSR above 90% while approaching direct refusal-vector steering.The method is attributed to learned refusal-direction reconstruction that steers malicious activations toward refusal-inducing regions.Source: §4.1 · p. 8

4.2 UTILITY PRESERVATION (RQ1)

• AlphaSteer maintains high utility across instruction following, safe questions, and mathematical reasoning tasks, whereas baselines show unstable degradation and CAST fails on the mathematical benchmarks.The comparison covers four utility benchmarks; CAST obtains 0% accuracy across MATH and GSM8K for all models.Source: §4.2 · p. 8, Table 2 · p. 8
• As steering strength increases, AlphaSteer raises DSR while keeping utility unaffected, unlike baselines whose utility declines as safety improves.The reported behavior is shown across steering strengths and defense success rates.Source: §4.2 · p. 9, Figure 4 · p. 9

4.3 MECHANISM (RQ2)

• PCA visualizations show benign activations remain largely unchanged while malicious activations shift toward a single direction as steering strength increases.This behavior supports the null-space mechanism for preserving benign activations while steering malicious ones.Source: Figure 3 · p. 8, §4.2 · p. 9

4.4 CASE STUDY (RQ3)

• In a ReNeLLM case study on Llama-3.1-8B-Instruct, AlphaSteer changes an unsafe malicious response into a refusal while retaining helpful responses to benign prompts.The AlphaSteer response matches the vanilla model on the benign prompt.Source: §4.4 · p. 9, Figure 5 · p. 9

• AlphaSteer’s effectiveness remains unknown on large reasoning models and larger models.Source: §5 · p. 10

6 CONCLUSION

• AlphaSteer constructs zero steering vectors for benign prompts through null-space projection and refusal directions for malicious prompts, targeting utility preservation and safety enhancement.The method is presented as theoretically grounded and learnable.Source: §6 · p. 10, §A.2 · p. 18
• AlphaSteer is evaluated using publicly available models and benchmark datasets without private or human-subject data.The experiments are described as relying solely on public resources.Source: ETHICS STATEMENT · p. 11
• Steering techniques could be misused to reconstruct a negative refusal vector that facilitates jailbreak or backdoor attacks.The paper identifies this risk during the linear-regression training process.Source: ETHICS STATEMENT · p. 11
• The authors state that no method offers absolute security despite AlphaSteer’s stronger defense against diverse jailbreak attacks.They call for continued attention to emerging threats and ethical deployment.Source: ETHICS STATEMENT · p. 11
• Activation steering methods improve safety against jailbreaks but must balance refusal behavior against utility on benign tasks.Prior approaches rely largely on heuristic calibration rules or manually crafted conditions.Source: §A.2 · p. 18

B METHODOLOGY

• AlphaSteer defines a benign activation matrix and uses the null space of its covariance to construct a projection matrix.The covariance is formed from benign activations, and SVD supplies eigenvectors associated with zero eigenvalues.Source: §B.1 · p. 19, §B.2 · p. 19, §2 · p. 20
• The benign constraint ensures a zero steering term for every benign activation, leaving benign activations unchanged.The projection-based construction satisfies the stated benign constraint.Source: §2 · p. 20
• The steering transformation is fitted by minimizing an optimization objective that maps projected malicious activations toward the refusal targets.The derivation reparameterizes the variables and obtains the solution through gradient stationarity and a pseudoinverse.Source: §B.3 · p. 20
• The final transformation is obtained by restoring the original variables after solving the reparameterized optimization problem.The substitutions identify the projected malicious activations, refusal targets, projection matrix, and trainable transformation.Source: §B.3 · p. 20

C.1 TRAINING COMPLEXITY OF ALPHASTEER

• The overall per-layer training complexity combines NbD2, NmD2, and D3 terms, and applying the procedure across L layers scales the offline cost accordingly.The terms reflect benign processing, malicious processing, and dimensional decomposition or linear-algebra operations.Source: §C.1 · p. 21
• In practice, AlphaSteer’s cost is dominated by either benign-covariance SVD or benign/malicious matrix multiplications, depending on Nb, Nm, and D.After precomputation, inference requires only one projection and update per layer.Source: §C.1 · p. 21
• CAST has the largest training cost among steering methods, while AlphaSteer and Jailbreak Antidote have comparable PCA/SVD-dominated costs and Surgical is lightest.All listed steering methods remain cheaper than gradient-based Circuit Breakers.Source: §C.2 · p. 22, Table 3 · p. 22
• AlphaSteer requires a single O(D2) linear transform per steered layer at inference, which remains negligible relative to a transformer forward pass.Surgical, CAST, and Jailbreak Antidote use O(D) vector operations per layer.Source: §C.2 · p. 22, Table 4 · p. 22

D.1 IMPLEMENTATION DETAILS

• Experiments use greedy decoding and report results across multiple model sizes, jailbreak DSR comparisons, and utility benchmarks.The implementation uses a single NVIDIA A40 GPU and an Intel Xeon Gold 6248R CPU with 96 cores.Source: §D.1 · p. 23, §E.1 · p. 26, Table 5 · p. 26
• Safety and utility are measured with GPT-4o using DSR for rejecting jailbreaks and CR for compliance on benign prompts.The evaluation prompts for these classifications are provided in Figure 6.Source: §D.1 · p. 23, Figure 6 · p. 24
• The test set contains 100 AdvBench prompts combined with various jailbreak methods, while data splits exclude identical or semantically similar test prompts.The setup is designed to reduce information leakage between training, validation, and testing.Source: §D.1 · p. 23, §D.2 · p. 25
• AlphaSteer uses 720 prompts to extract refusal vectors, 14,000 benign prompts for the null-space matrix, and 2,000 malicious prompts for fitting the transformation.These datasets combine AdvBench, MaliciousInstruct, TDC23-RedTeaming, JailbreakBench, WildGuardMix, JailbreakV-28K, Alpaca, Coconot, and MATH data.Source: §D.1 · p. 23
• Evaluation attacks are excluded from malicious training data, and the average BLEU similarity between training and evaluation prompts is 0.050, with 98% of test prompts below 0.3.The authors interpret this as evidence of low textual overlap and generalization to unseen attacks.Source: §D.1 · pp. 23–24

Comparison with refusal training

• AlphaSteer yields better results with lower training time than refusal training on the same amount of data.The comparison uses the same data quantity, and AlphaSteer’s training time is reported as much lower than SFT.Source: §E.2 · p. 26, Table 9 · p. 26

Ablation

• Linear regression outperforms a two-layer MLP for AlphaSteer, which the authors attribute to possible MLP overfitting.The MLP may be over-parameterized relative to the regression-based construction.Source: §E.3 · p. 27, Table 10 · p. 27

Generalization to unseen domains

• AlphaSteer preserves utility when math-related training data is removed, indicating generalization to unseen math domains.The average PEC remains 0.92 without math data, compared with 0.97 when math data is included.Source: §E.4 · p. 27, Table 12 · p. 27

Generalization to more malicious prompts

• AlphaSteer achieves strong safety performance across AdvPrompter, FlipAttack variants, and WildGuard Test at λ = 0.5.It attains 100% DSR on AdvPrompter, 84–100% across FlipAttack variants, and 100% on WildGuard Test.Source: §E.5 · p. 28, Table 13 · p. 28

Adaptive GCG robustness

• 95.5% DSR and final loss 1.78 show AlphaSteer remains most robust under 200-step adaptive GCG attacks.The vanilla model reaches 57.5% DSR with loss 1.28, while RV Steer reaches 79.5% DSR with loss 1.55.Source: §E.6 · p. 28, Figure 7 · p. 28
• AlphaSteer’s dynamic steering forces GCG to overcome more complex geometry, slowing convergence and maintaining higher residual loss than fixed-direction steering.This mechanism is offered as the explanation for its stronger post-attack safety.Source: §E.6 · p. 28

Activation dynamics

• Benign activations remain largely unaffected while malicious activations are steered toward refusal across visualized models and attack cases.The same pattern appears in PCA visualizations and the ReNeLLM case study.Source: §F.1 · p. 29, Figure 9 · p. 29, §F.2 · p. 29

Performance trends

• As steering strength increases, AlphaSteer largely preserves utility while baseline methods tend to harm or preserve utility inconsistently.Across individual tasks and aggregate utility-versus-DSR trends, all methods increase DSR, but AlphaSteer shows better utility preservation.Source: §F.5 · p. 30, §F.6 · p. 31, Figure 15 · p. 32

Space coverage

• PEC measures the fraction of benign test activation energy contained in the benign training subspace, linking coverage to utility preservation.If test benign activations lie within the training span, AlphaSteer projects them toward a zero steering vector and leaves them unaffected.Source: §F.7 · p. 33
• Llama3.1 achieves PEC scores of 0.959–0.993 across layers with k = 1638 ≈ 0.4D, indicating strong activation-space coverage.Here, PEC = 1 means all activations lie in the space spanned by training activations.Source: §F.7 · p. 33

Effect of benign data size

• Reducing benign training data lowers PEC and utility preservation, while safety enhancement remains largely unaffected.The results attribute the utility drop mainly to decreased space coverage.Source: §F.7.1 · p. 33, Figure 16 · p. 33, Table 14 · p. 33

Null-space structure

• Eigenvalues rapidly approach zero, supporting the existence of a benign null space for the constrained steering transformation.This observation provides empirical support for the null-space construction.Source: §F.8.1 · p. 34, Figure 17 · p. 34

Null-space threshold

• Increasing p% raises DSR but degrades utility because shrinking the preserved benign subspace reduces coverage of benign activations.The norm distributions also become more separable for benign and malicious prompts at deeper layers and larger p%.Source: §F.8.2 · p. 35, Table 16 · p. 35, Table 17 · p. 35

Evaluator mismatch

• LlamaGuard-3-8B reports higher DSR than GPT-4o, but can misclassify compliance responses as safe.The discrepancy is especially noted for Cipher.Source: G IMPACT OF EVALUATION MODEL CHOICE · p. 42, Table 18 · p. 42
• GPT-4o flags a jailbreak when the model complies, while LlamaGuard misclassifies the same surface-benign response as safe.The case study attributes the mismatch to LlamaGuard inspecting surface content rather than requiring explicit refusal.Source: G IMPACT OF EVALUATION MODEL CHOICE · p. 42, Figure 24 · p. 43