CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment

Introduction

• Global safety tuning can improve refusal behavior but also alter benign, safety-adjacent responses, causing over-refusal or degraded reasoning—the alignment tax CLEAR aims to reduce.LLMs’ capabilities across reasoning, coding, and instruction following enable open-ended deployment but also bring risks including jailbreaks, harmful generation, and adversarial prompt manipulation.Source: Introduction · p. 1
• CLEAR freezes the backbone and jointly optimizes a safety LoRA with a learned latent gate that continuously scales intervention by prompt risk, rather than filtering outputs post hoc.Subtype-aware gate weighting and a hard pairwise margin objective separate harmful and benign prompts in latent gating space; benign prompts receive little intervention while harmful or adversarial prompts receive stronger activation.Source: Introduction · p. 1, Figure 1 · p. 2
• 0.50% HarmBench ASR: CLEAR lowers ASR to this level, versus 15.50%–24.00% for other aligned Llama-3 and Alpaca variants, while maintaining competitive utility.On Llama-3-8B-Instruct, CLEAR retains 73.46% GSM8K accuracy, about 7 percentage points above SFT and standard LoRA. On Gemma-2-2B-it, GSM8K improves from around 38% under SFT/LoRA to 42%, and TruthfulQA MC2 from around 47% to 53%.Source: Introduction · p. 2

Related Work

• Safety alignment commonly uses SFT, RLHF, and preference optimization to reduce harmful generations, but broad objectives can degrade utility and cause over-refusal on benign safety-adjacent prompts (Ouyang et al. 2022; Touvron et al. 2023; Bai et al. 2022; Rafailov et al. 2023).Related approaches mitigate the trade-off with improved preference data, calibrated refusal policies, or external moderation systems.Source: Related Work · p. 2
• LoRA-based safety methods typically apply adapters uniformly, whereas prior routed computation targets efficiency or task specialization; CLEAR continuously gates a safety LoRA from frozen prompt representations (Hu et al. 2022; Qi et al. 2024; Hsu et al. 2024; Wu et al. 2025; Shazeer et al. 2017; Fedus, Zoph, and Shazeer 2022; Pfeiffer et al. 2021; Huang et al. 2023).Source: Related Work · p. 2
• Prior work probes or steers safety-related representations and uses external guardrails, while CLEAR uses frozen internal hidden states to continuously control adapter strength (Zou et al. 2023; Arditi et al. 2024; Rocchetti, Ferrara et al. 2024; Cao, Yang, and Zhao 2025; Li et al. 2025).Representation-level signals have been used for analysis, steering, or external filtering rather than continuous modulation of a safety adapter.Source: Related Work · pp. 2–3

Background

• Safety alignment seeks to preserve utility on benign prompts while reducing harmfulness on unsafe or adversarial prompts, but global updates apply the same modified policy to both prompt types.CLEAR instead estimates an input-conditioned intervention strength: low-risk prompts stay close to the frozen backbone, while high-risk prompts receive stronger safety adaptation, aiming to reduce unnecessary refusals.Source: Background · p. 3

Overview

• CLEAR freezes the pretrained LLM and scales a safety LoRA branch with a continuous hidden-state gate, keeping benign prompts closer to the backbone while enabling stronger intervention for higher-risk prompts.The gate score gi ranges from 0 to 1 and modulates the adapter update inside the forward computation, rather than acting as a post-hoc classifier or binary refusal switch.Source: Overview · p. 3

Gate: Prompt-Level Risk Scoring

• CLEAR scores prompt-only hidden states with a sigmoid MLP trained on safe/unsafe labels, upweighting adversarial subtypes and adding pairwise margins so unsafe prompts score above safe ones.The gate uses WildJailbreak subtype-aware binary cross-entropy and emphasizes confusing pairs, including high-scoring safe prompts and low-scoring unsafe prompts.Source: Gate: Prompt-Level Risk Scoring · pp. 3–4

LoRA: Risk-Conditioned Safety Adapter

• The safety LoRA branch is trained on unsafe prompt-completion pairs, while the same gate score controls adapter strength across adapted modules and the frozen backbone remains unchanged.The gate learns intervention strength through binary and pairwise losses, while the LoRA branch learns safe aligned behavior from target responses to unsafe prompts.Source: LoRA: Risk-Conditioned Safety Adapter · p. 4

Optimization Procedure

• Training jointly updates the gate and LoRA parameters while freezing the backbone, using gate losses for intervention strength and adapter alignment loss on unsafe examples.Safe target responses contribute only to the adapter alignment loss for unsafe examples.Source: Optimization Procedure · p. 4

• The baseline comparison asks whether CLEAR improves the safety–utility trade-off over globally applied safety tuning and existing safety-aligned variants.This comparison is framed alongside the evaluation’s safety and utility benchmarks.Source: Experimental Evaluation · p. 4
• The evaluation tests whether CLEAR’s learned hidden-state gate routes reliably and efficiently compared with external guard models.Source: Experimental Evaluation · p. 4
• The evaluation asks whether CLEAR’s benefits persist across model scales and whether it can defend against latent adaptive attacks.Source: Experimental Evaluation · p. 4
• CLEAR is evaluated on HarmBench ASR, XSTest safe over-refusal and unsafe refusal rates, and GSM8K, MMLU, and TruthfulQA accuracy (MC1 and MC2).These benchmarks cover safety and utility.Source: Experimental Evaluation · p. 4

3. Adapter alignment

• Each adapted projection layer uses h′ ← (W + g_iΔW_θ)h, scaling the adapter update by the gate.Source: §3 · p. 5
• The language-model loss is computed on B_unsafe, the batch subset whose labels y_i equal 1.Source: §3 · p. 5

Experimental Setting

• Experiments use Gemma-2-2B-it and Llama-3-8B-Instruct, a shared WildJailbreak training split, greedy decoding, and medians over three independent runs unless otherwise stated.SFT updates backbone parameters; standard LoRA uses CLEAR’s LoRA configuration with the adapter always active, whereas CLEAR routes it through a learned hidden-state-conditioned gate.Source: §4 · p. 5

Update

• CLEAR’s gate is a two-layer MLP with hidden dimension one eighth of the backbone hidden size, sigmoid output, and hidden-state inputs from three intermediate and later transformer layers.Layer choices, hyperparameters, model-specific settings, hardware, training cost, and additional gate and objective ablations are provided in appendices.Source: §4 · p. 5

Safety-Utility Trade-off Analysis

• On Llama-3-8B-Instruct, CLEAR lowers HarmBench ASR from 32.25% to 0.50%, raises XSTest unsafe refusal from 88.50% to 92.50%, and retains 73.46% GSM8K accuracy versus 66.34% for SFT and 66.72% for standard LoRA.CLEAR is not best on every metric: standard LoRA reaches 0.00% HarmBench ASR, while the base model has higher MMLU accuracy. On Gemma-2-2B-it, CLEAR reaches 0.00% HarmBench ASR and 96.00% unsafe refusal, while safe over-refusal rises to 9.60%.Source: §4 · p. 5, Table 1 · p. 6

Comparison with Safety-Aligned Model Families

• CLEAR lowers HarmBench ASR from 32.25% to 0.50% from Llama-3-8B-Instruct and from 22.50% to 0.50% from P-SACPO in the Alpaca family.The Alpaca-family result is labeled P-SACPO + CLEAR because CLEAR is applied on top of P-SACPO; conditional routing retains utility close to each corresponding initialization.Source: §4 · p. 5, Table 2 · p. 6

Gate Quality Analysis

• On XSTest, CLEAR’s hidden-state gate offers a favorable routing trade-off and is substantially smaller than external guards; PromptGuard routes nearly all prompts as unsafe under the evaluated label mapping.The analysis uses a 0.5 threshold for CLEAR; PromptGuard’s behavior reflects a mismatch between prompt-injection detection and harmful-versus-benign routing, not necessarily intrinsic over-conservatism.Source: §4 · pp. 5–6, Figure 2 · p. 5, Table 3 · p. 6

Scaling Analysis

• Across Qwen2.5-Instruct sizes from 0.5B to 7B, CLEAR reduces HarmBench ASR to at most 2.00%; gate ROC-AUC rises from 0.78 to 0.96 and PR-AUC from 0.75 to 0.93.Utility is largely preserved; Qwen-7B retains GSM8K exactly and slightly improves MMLU. The largest unsafe-safe gate-score margin occurs at 7B.Source: §4 · p. 6, Table 4 · p. 7

Latent Space Adaptive Attack

• Under adaptive hidden-state attacks at embedding, middle, and final layers, CLEAR’s ASRs are 0.0%, 7.1%, and 18.4%, versus 54.0%, 48.0%, and 31.0% for LoRA and 47.0%, 35.0%, and 22.0% for Full SFT.Perturbed states are projected onto a clean PCA subspace estimated from benign prompts; the figure reports Gemma results at perturbation budget ϵ = 0.001.Source: §4 · pp. 6–7, Figure 3 · p. 7

Conclusion

• Across multiple safety and utility benchmarks, CLEAR improves harmful-prompt robustness while preserving more general-purpose capability than globally applied safety tuning.The conclusion describes this as a substantial improvement but notes that the benefit depends on reliable routing.Source: Conclusion · p. 7
• CLEAR uses a hidden-state gate to modulate a safety-specific LoRA adapter according to prompt risk, with subtype-aware optimization and a hard pairwise margin objective separating benign and malicious inputs.This conditional approach avoids globally modifying the backbone.Source: Conclusion · p. 7
• Benign prompts may trigger unnecessary intervention, while unsafe prompts with low gate scores may bypass the adapter, making gate calibration and distribution-shift robustness future priorities.The conclusion identifies reliable routing as a condition for CLEAR’s benefit.Source: Conclusion · p. 7

Limitations

• CLEAR depends on reliable gating, may fail under distribution shifts, and is evaluated primarily on small open-weight models and single-turn text safety.Benign prompts may trigger unnecessary interventions, unsafe prompts with low gate scores may bypass the adapter, and unseen jailbreaks may circumvent the safety branch; long-horizon deception and multi-turn adversarial interactions are outside scope.Source: Limitations · p. 10

Reproducibility Details

• The experiments use google/gemma-2-2b-it and meta-llama/Meta-Llama-3-8B-Instruct as the Gemma-2-2B-it and Llama-3-8B-Instruct checkpoints.The authors suggest more robust gating, adaptive routing, and larger-scale evaluations under realistic deployment settings as future work.Source: Reproducibility Details · p. 10, Limitations · p. 10
• CLEAR training uses 261,559 WildJailbreak examples: 128,781 safe and 132,778 unsafe, across vanilla_benign, adversarial_benign, vanilla_harmful, and adversarial_harmful subtypes.The data are from the allenai/wildjailbreak training split.Source: Reproducibility Details · p. 10
• SFT, standard LoRA, and CLEAR use the same WildJailbreak data for one epoch; SFT and CLEAR use micro-batch size 4 with 16 accumulation steps, while LoRA uses 16 and 4.Both configurations yield an effective batch size of 64.Source: Reproducibility Details · p. 10
• For each backbone, the CLEAR gate averages hidden states from three intermediate and later transformer layers.Table 6 summarizes the selected layers.Source: Reproducibility Details · p. 10
• XSTest and HarmBench use greedy decoding with max_new_tokens=150 and 512, respectively; GSM8K and MMLU are 5-shot, while TruthfulQA is 0-shot.XSTest uses string-match refusal classification, HarmBench uses the cais/HarmBench-Llama-2-13b-cls classifier, and utility evaluations use EleutherAI lm-evaluation-harness.Source: Reproducibility Details · p. 10
• 5.51 × 10^6 trainable parameters for CLEAR versus 8.03 × 10^9 for SFT on Llama-3-8B-Instruct; CLEAR uses less than half the memory footprint.CLEAR adds modest training time and trainable parameters relative to standard LoRA, but remains significantly more efficient than full-model finetuning; similar trends hold for Gemma-2-2B-it.Source: Reproducibility Details · p. 10
• Experiments ran on a single NVIDIA GH200 120GB GPU.This specifies the hardware used for the experiments.Source: Reproducibility Details · p. 10

Gate Architecture and Layer Selection Ablation

• Using multiple layers with the MLP gate lowers HarmBench ASR from 0.25% to 0.00% and raises GSM8K from 34.42% to 41.93%.The multi-layer Transformer also improves over its single-layer variant on HarmBench ASR, unsafe refusal, safe over-refusal, and GSM8K, but does not consistently outperform the multi-layer MLP.Source: Gate Architecture and Layer Selection… · p. 10

Training Objective Ablation

• BCE-only routing raises HarmBench ASR from 0.00% to 3.25%, lowers unsafe refusal from 96.00% to 83.50%, and raises safe overrefusal to 14.40% versus full CLEAR.The full CLEAR objective improves harmful-instruction robustness and unsafe refusal while preserving stronger utility on GSM8K and TruthfulQA.Source: Training Objective Ablation · p. 10
• Removing LoRA L2 regularization slightly worsens HarmBench ASR, safe overrefusal, and GSM8K, suggesting regularization helps stabilize the adapter and safety–utility balance.Source: Training Objective Ablation · p. 10

Latent Space Adaptive Attack

• The targeted PGD attack adds a trainable perturbation to an intermediate hidden state and optimizes it to increase the likelihood of a target harmful response under an ℓ∞ constraint.The perturbation is applied before the model’s remaining layers, and projected gradient descent keeps it within the specified constraint set.Source: Latent Space Adaptive Attack · pp. 10–11
• CLEAR builds a clean latent subspace by applying weighted PCA to benign-prompt hidden states, then softly projects inference-time activations toward that subspace.The gate score weights benign token representations; projection strength γ∈[0,1] controls the interpolation, with smaller values enforcing stronger projection.Source: Latent Space Adaptive Attack · p. 11
• The defense adapts projection strength to gate-score anomalies: its sigmoid transition spans approximately a∈[0.2,0.8], strengthening projection for unusually high scores while minimally altering benign prompts.The sigmoid inflection point is a=0.5, and the adaptive projection is intended to respond to clearly harmful inputs while remaining robust to small gate-score fluctuations.Source: Latent Space Adaptive Attack · pp. 11–12
• Evaluation uses 100 HarmBench harmful prompts on Gemma-2-2B-it, with white-box PGD at embedding, middle, and final layers using ϵ=0.001 and 10 steps.The gate-adaptive PCA defense estimates a 128-dimensional benign-reference subspace for each target layer and dynamically sets γ∈[0.05,0.80].Source: Latent Space Adaptive Attack · p. 12

Training Dynamics

• Figure 4 shows the LoRA language modeling, gate BCE, and hard-pair margin losses decreasing as the adapter learns aligned responses and the gate learns a classification boundary with a larger margin.Together, the losses track adapter training, gate classification, and separation between safe and unsafe prompts.Source: Training Dynamics · p. 12
• As training progresses, unsafe prompts receive higher gate scores and safe prompts lower scores, with clear separation on Llama-3-8B-Instruct and Gemma-2-2B-it.The curves support using the gate to activate the safety adapter primarily for unsafe inputs while suppressing unnecessary activation on benign prompts.Source: Training Dynamics · p. 12

HarmBench Evaluation

• HarmBench ASR is the fraction of greedy-decoded generations judged yes for an unambiguous, non-minimal harmful behavior by the official classifier.Generations use the target model’s chat template and max_new_tokens=512.Source: HarmBench Evaluation · pp. 12–13

XSTest Evaluation

• XSTest reports unsafe refusal rate and safe-prompt over-refusal rate using a string-match refusal classifier.A completion counts as refusal when its lowercased, stripped text begins with a standard refusal phrase; otherwise it counts as full compliance.Source: XSTest Evaluation · p. 13

Utility Evaluation

• Utility evaluation uses lm-eval-harness to measure GSM8K exact match, MMLU accuracy, and TruthfulQA MC1/MC2 accuracy.GSM8K and MMLU use 5-shot prompting; TruthfulQA uses 0-shot prompting.Source: Utility Evaluation · p. 13