Source-linked AI summary
Does Deeper Reasoning Compromise Alignment? Revealing and Mitigating of Alignment Collapse in Large Reasoning Models
Yu-Hang Wu, Yu-Jie Xiong, Henghua Zhang, Bairui Zhang, Jia-Chen Zhang, Shaohua Li
TL;DR
The paper asks whether extended reasoning destabilizes alignment under perturbation, a question left underexplored despite widespread confidence in reasoning models’ safety. It measures this vulnerability, develops an attack and mechanism analysis, and proposes a lightweight defense. The results identify Alignment Collapse, with rising ALR and substantially worsened safety robustness under reasoning-amplified attacks.
Problem
Whether extended reasoning itself systematically destabilizes alignment robustness under external perturbations remains underexplored.
Method
The paper quantifies alignment degradation with ALR, uses RT to induce extended reasoning and amplify attacks, analyzes Attention Dilution, and proposes training-free RRA.
Results
Reasoning depth produces a monotonic rise in ALR across tested models, while RT drives severe safety degradation, including a 63.28% drop for Qwen3-8B.
Takeaways & Limitations
Extended reasoning can increase vulnerability to external perturbations, and residual re-emphasis of the input is proposed to restore alignment robustness without training.
Takeaways & Limitations
The evaluation covers multiple models and benchmarks but not all modern LRMs, perturbation types, or real-world interactions, and the mechanistic analysis may omit other causes.
Abstract
from arXiv · showhide
The emergence of Chain-of-Thought (CoT) has established a robust foundation for Large Reasoning Models (LRMs). While deep reasoning is widely believed to enhance safety alignment, the stability of alignment mechanisms under extended reasoning remains underexplored. This paper challenges the prevailing view by revealing a critical vulnerability: Deep Reasoning May Induce Alignment Collapse. To rigorously quantify this phenomenon, we propose the Alignment Loss Rate (ALR) metric. Our experiments demonstrate that as reasoning depth increases, ALR rises significantly, indicating a severe degradation in model robustness against external perturbations. Capitalizing on this instability, a novel jailbreaking paradigm, Reasoning Trap (RT), is proposed. RT induces the model into extended reasoning to amplify the impact of adversarial attacks, leading to a sharp decline in safety capabilities. To elucidate the mechanism behind this collapse, we identify Attention Dilution as the root cause, arising from the competition for attention between the extended reasoning process and the original input. To mitigate this, Reasoning Residual Alignment (RRA), a lightweight defense strategy that dynamically re-emphasizes the input via residual connections integrated with the reasoning process.
1 Introduction
The paper examines whether extended reasoning undermines alignment robustness under perturbation. It identifies Alignment Collapse, explains it through Attention Dilution, and proposes RT and RRA to expose and mitigate the vulnerability.
- CoT improves complex-task performance and helped establish Large Reasoning Models built around extended internal reasoning.
- Existing work leaves limited mechanistic evidence about whether extended reasoning itself systematically destabilizes alignment.
- Alignment Collapse describes the degradation of adherence to original constraints when deep reasoning faces external perturbations.
- As reasoning depth increases, ALR shows systematic deviation from intended alignment and increasing fragility.
- RT amplifies adversarial attacks by inducing extended reasoning, while RRA re-emphasizes the input through training-free residual connections.
- Attention Dilution attributes the collapse to competition between extended reasoning and the original input, causing the input’s attention weights to decay rapidly.
2 Related Work
Related work frames reasoning as either inference-time scaffolding or training-based enhancement, while prior jailbreaking research spans manual, systematic, and gradient-based attacks.
- Inference-time methods such as CoT and Tree of Thoughts elicit multi-step reasoning without changing model weights.
- Training-based approaches form the second major paradigm for improving reasoning capabilities.
- Jailbreaking research includes manual role-playing techniques such as DAN and systematic classifications of attack tactics and objectives.
- Gradient-based attacks including GCG, AutoDAN, and I-GCG optimize prompts iteratively to bypass safeguards.
3 The Alignment Issues in LRM
The section evaluates whether deeper reasoning improves task performance while weakening alignment robustness under external perturbations. It introduces controlled comparisons across reasoning depths, models, datasets, and safety settings, using ALR to quantify proportional degradation.
- Experimental setup: Controlled experiments compare non-reasoning and deep-reasoning modes using Qwen3 on AIME2024 and LogicAsker datasets.L = 0 denotes non-reasoning, while L = 2048 and L = 4096 denote increasing reasoning depths.
- Measurement: ALR measures percentage degradation relative to original capability, with higher values indicating greater alignment compromise under perturbations.The metric compares performance on clean and perturbed inputs at each reasoning depth.
- Alignment robustness results: Qwen3-8B’s clean accuracy increases from 20.83% at L = 0 to 31.67% at L = 4096, while its Perturbation-II drop grows from 6.67% to 12.51%.The results show simultaneous task-performance gains and increasing degradation under perturbation.
- Alignment robustness results: Qwen3-14B’s ALR rises from 9.2% at L = 0 to 20.8% at L = 2048 and 33.1% at L = 4096.The reported trend is monotonic across increasing reasoning depth.
- Conclusion: The paper names this depth-dependent vulnerability Alignment Collapse: deeper reasoning improves task performance but compromises robustness under identical perturbations.The phenomenon is described as increasing susceptibility to interference as reasoning extends.
4 Reasoning Trap: Uncovering Safety Risks in Deep Reasoning
Reasoning Trap (RT) combines existing jailbreak perturbations with a trigger that elicits extended reasoning, exposing reasoning-amplified safety degradation. Across attacks and models, deeper reasoning reduces rejection robustness and can amplify adversarial effects.
- Purpose: RT simulates extended reasoning in standard LLMs to test whether reasoning amplifies adversarial perturbations and weakens refusal behavior.The framework integrates the reasoning trigger with existing attacks rather than replacing them.
- Empirical results: Deep reasoning lowers safety robustness: Qwen3-8B’s FlipAttack RSR falls from 92.31% without reasoning to 57.42% with reasoning.The comparison uses L = 0 for standard generation and L = 4096 for reasoning mode.
- RT framework: RT combines a perturbation function applying existing jailbreak attacks with a reasoning trigger that elicits extended intermediate reasoning.The resulting output is generated from the perturbed input and trigger through prompt concatenation.
- Empirical results: 63.28%: RT combined with FlipAttack causes Qwen3-8B’s RSR to plummet by 63.28%.Under ArtPrompt, Llama2-7B and Qwen3-14B show RSR declines of 43.65% and 26.54%, respectively.
- Reasoning-depth effect: With FlipAttack and ArtPrompt, the RSR drop increases monotonically with reasoning depth, whereas No Attack remains stable under RT-induced reasoning.This pattern places models in an alignment collapse zone as reasoning depth increases.
5 Experiment
The experiments evaluate reasoning-related alignment degradation across reasoning and safety benchmarks, using attack baselines and RT-integrated variants. Results show that increasing reasoning depth raises ALR and makes safety alignment more fragile.
- Benchmarks: The experiments use AIME2024 and LogicAsker for reasoning evaluation and AdvBench’s 520 malicious prompts for safety evaluation.Safety comparisons include Clean, Perturbation-I, and Perturbation-II settings.
- Experimental design: The safety study compares FlipAttack, ArtPrompt, and PAP with their RT-integrated variants to assess degradation under simulated deep reasoning.RSR is used as the primary safety metric.
- Alignment Collapse: Increasing reasoning depth L produces a monotonic rise in ALR across all tested models.At maximum depth, Qwen3 models show severe accuracy degradation under perturbation relative to the non-reasoning baseline.
- Results and discussion: Reasoning modes are more attack-susceptible than non-reasoning modes, with Qwen3-8B’s FlipAttack rejection rate dropping from 92.31% to 57.42%.RT further magnifies this instability, including a 63.28% Qwen3-8B drop.
6 Interpretable Analysis and Defending Measure
The paper attributes alignment collapse to Attention Dilution: longer reasoning competes with the initial input for limited attention. It proposes RRA, which re-injects the input through a residual connection to counteract this effect.
- Interpretable analysis: As reasoning depth increases, generated tokens structurally compete with the original input for a fixed attention capacity.The analysis models attention allocation between the initial input X and generated reasoning Y.
- Defending measure: RRA is evaluated as a defense-performance comparison at reasoning depth L = 4096 against the baseline reasoning mode without RRA.Table 6 reports restoration of safety alignment capabilities after integrating RRA.
- Attention Dilution: Positional attenuation makes models attend disproportionately to proximal reasoning tokens while neglecting distant safety instructions.This effect grows as the reasoning sequence becomes longer.
- Attention Dilution: Attention Dilution compresses attention weights assigned to fixed input tokens, marginalizing early safety instructions and increasing perturbation vulnerability.The mechanism follows from softmax competition as the reasoning contribution enlarges.
- Defending measure: RRA restructures generation from X → Y → Response into [X; Y; X], re-injecting the original input after reasoning.The residual shortcut is designed to counteract attention dilution.
7 Conclusion
The paper concludes that deeper reasoning can increase vulnerability to external perturbations, exposes this risk with RT, analyzes Attention Dilution as a possible mechanism, and proposes RRA as a lightweight training-free mitigation.
- Conclusion: The paper reveals Alignment Collapse, introduces RT to expose reasoning-amplified safety risks, analyzes Attention Dilution, and proposes RRA as a lightweight training-free mitigation.These contributions address the impact of deep reasoning on alignment robustness.
Limitations
The study uses controlled evaluations across selected models, datasets, reasoning depths, and perturbation types, while acknowledging that these settings do not cover all LRMs or real-world interactions.
- The evaluated systems and perturbations do not exhaust the full space of modern LRMs or real-world user interactions.The authors identify broader model families, perturbation types, and user settings as directions for future evaluation.
Ethical Statement
The paper frames its work as defensive red teaming intended to uncover safety risks and develop robust mitigations. Its experiments use selected datasets, models, and generated perturbations to evaluate alignment robustness.
- The authors state that their goal is defensive red teaming and robust mitigation rather than facilitating offensive attacks.They emphasize responsible disclosure and advancing LLM safety.
- Qwen3 models are used for Alignment Collapse analysis because they support switching between standard and reasoning modes and expose internal states.These properties support controlled comparisons and mechanistic attention analysis.
- AIME2024 and LogicAsker are used to assess alignment robustness in mathematical and logical reasoning contexts.LogicAsker is added to strengthen the robustness of the alignment evaluation.
- DeepSeek-V3 generates two categories of adversarial inputs from clean baselines for systematic evaluation of deep reasoning effects.
- Perturbation-I inserts a grammatically correct but useless, same-topic sentence immediately before the final query.The design tests whether extended reasoning amplifies susceptibility to external perturbation.
B Additional Experiments
Additional experiments test whether Alignment Collapse generalizes across datasets and models, assess statistical reliability, examine Reasoning Trap, and analyze Attention Dilution and RRA variants.
- Cross-Dataset Validation: Deepening reasoning systematically undermines alignment robustness across both AIME2024 and LogicAsker.Cross-dataset results for Qwen3 models show consistent behavior under clean and perturbed conditions.
- Cross-Model Validation: GLM4.6 clean-input accuracy rises from 98.20% at L=0 to 100.00% at L=4096, while its Perturbation-I accuracy drop expands from 0.10% to 0.30%.
- Reasoning Trap: Reasoning Trap increases generated tokens and can sharply reduce Rejection Success Rate when activating deep reasoning during attacks.For Qwen3-8B with FlipAttack, average generated tokens rise from 282 to 1568.
- Attention Dilution: Attention Dilution is analyzed through RoPE, where increasing reasoning depth enlarges the distance between current reasoning tokens and initial safety instructions.The analysis connects positional effects and locality bias to weakened attention toward distant constraints.
- Asymptotic Collapse of Attention Weights: As reasoning length increases, RoPE-related numerator decay and denominator inflation reduce the attention weight assigned to initial alignment context.The derivation states that, without residual interventions such as RRA, attention to safety constraints is systematically eroded.
- RRA Variants: RRA re-injects the original input after the reasoning trace, whereas alternative variants append either a fixed safety reminder or an input-dependent anchor.The variants test whether gains arise from an explicit safety cue or restored salience of the original input.
- RRA Variants: The fixed safety reminder slightly improves refusal-oriented behavior but its effect does not generalize to ordinary reasoning tasks.RRA is presented as the input-reusing alternative that restores attention to the original constraints.
D.2 RRA Variants on Reasoning Benchmarks
The reasoning-benchmark ablation compares a fixed safety reminder, an input-dependent anchor, and RRA under perturbations on AIME2024 and LogicAsker. RRA performs best across these ordinary reasoning evaluations.
- A generic safety reminder may distract models from AIME2024 and LogicAsker because these benchmarks require solving rather than refusing requests.
- The fixed safety reminder slightly reduces accuracy on both AIME2024 and LogicAsker under perturbation.Its benefit is specific to refusal-oriented evaluation rather than ordinary reasoning benchmarks.
- Both A(X) and RRA improve perturbed-task performance, with RRA consistently achieving the strongest gains.These results support restoring the salience of the original input rather than adding a generic safety instruction.
- RRA reuses the original input, adds no new safety-specific template, and improves both refusal robustness and perturbation robustness.The ablation frames RRA as less prompt-specific than the fixed safety reminder.
- At L=4096, the variants compare Base=[X; Y], +S=[X; Y; S], +A(X)=[X; Y; A(X)], and +X=[X; Y; X].The tables report accuracy changes relative to Base; +X reuses the original input without additional prompt tokens.