PaperlayerView exact revision on arXiv

Source-linked AI summary

ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack

Hao Li, Yankai Yang, G. Edward Suh, Ning Zhang, Chaowei Xiao

arXiv:2601.10173v1cs.CRcs.AIcs.CL

TL;DR

Indirect prompt injections can hijack LLM agents through malicious instructions in external data. ReasAlign uses structured reasoning and test-time judge-guided trajectory selection to preserve user intent. Across its evaluations, it reports stronger security and utility trade-offs than Meta SecAlign, including 94.6% utility on CyberSecEval2 versus 56.4%.

  • Problem

    Malicious instructions embedded in external data can hijack agent behavior while agents perform user tasks.

    Source: §1 · p. 1

  • Method

    ReasAlign reasons through the user query and conflicting external instructions, then uses a preference-trained judge to score reasoning steps and select a trajectory.

    Source: §1 · p. 2, §4.2 · p. 5

  • Results

    Across instruction-following and agentic workflow benchmarks, ReasAlign outperforms Meta SecAlign on security and reports 94.6% utility on CyberSecEval2 versus Meta SecAlign's 56.4%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The evaluations support ReasAlign as a practical defense that maintains high utility while improving security across diverse tasks.

    Source: §6 · p. 8

    Reasoning-based approaches introduce additional overhead, and the authors identify selectively varying reasoning length by task complexity and security risk as future work.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Models (LLMs) have enabled the development of powerful agentic systems capable of automating complex workflows across various fields. However, these systems are highly vulnerable to indirect prompt injection attacks, where malicious instructions embedded in external data can hijack agent behavior. In this work, we present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks. The core idea of ReasAlign is to incorporate structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks to defend against indirect injection attacks. To further ensure reasoning logic and accuracy, we introduce a test-time scaling mechanism with a preference-optimized judge model that scores reasoning steps and selects the best trajectory. Comprehensive evaluations across various benchmarks show that ReasAlign maintains utility comparable to an undefended model while consistently outperforming Meta SecAlign, the strongest prior guardrail. On the representative open-ended CyberSecEval2 benchmark, which includes multiple prompt-injected tasks, ReasAlign achieves 94.6% utility and only 3.6% ASR, far surpassing the state-of-the-art defensive model of Meta SecAlign (56.4% utility and 74.4% ASR). These results demonstrate that ReasAlign achieves the best trade-off between security and utility, establishing a robust and practical defense against prompt injection attacks in real-world agentic systems. Our code and experimental results could be found at https://github.com/leolee99/ReasAlign.

1 Introduction

Indirect prompt injections can hijack agents through malicious instructions embedded in external data, while rigid defenses may also suppress helpful instructions. ReasAlign uses structured reasoning and test-time trajectory selection; evaluations report stronger security and utility trade-offs than prior guardrails.

1 Introduction

  • Malicious instructions embedded in third-party data can enter an agent's context and hijack it during a user's task.The paper gives an example of a product review directing an agent to visit an attack site and enter credit-card information.Source: §1 · p. 1
  • Internal defenses such as Meta SecAlign can rigidly suppress external instructions, including helpful ones needed for open-ended tasks.The introduction identifies this as an overkill issue for Meta SecAlign.Source: §1 · p. 2
  • ReasAlign analyzes the user query, identifies conflicting injection instructions, preserves the user's original intent, and uses a judge to select the best reasoning trajectory.The preference-optimized judge scores individual reasoning steps at test time.Source: §1 · p. 2
  • 94.6% utility on CyberSecEval2, compared with 56.4% for Meta SecAlign, alongside stronger security across instruction-following and agentic workflow benchmarks.ReasAlign's utility is comparable to the undefended model without attacks and exceeds it under attack.Source: §1 · p. 2

2 Related Works

The paper reviews system-level and model-level prompt-injection defenses, noting trade-offs in deployment complexity and retained task utility. It also describes reasoning methods that use stepwise analysis, multiple paths, or task decomposition.

2.1 Prompt Injection Defense

  • System-level defenses use policies or flow constraints, but sophisticated policies and complex designs can substantially increase deployment costs.Examples include execution environment isolation, information-flow control, and dynamic policy updates.Source: §2.1 · pp. 2–3
  • External model-level guardrails detect injection attempts but may refuse responses, while internal guardrails can rigidly suppress helpful external instructions and limit open-ended task utility.Internal approaches discussed include StruQ, SecAlign, and Meta SecAlign.Source: §2.1 · p. 3

2.2 LLM Reasoning

  • Reasoning methods are grouped into step-by-step reasoning, multi-path exploration, and decomposition-based methods.The cited examples include chain-of-thought, tree- or graph-structured trajectories, and breaking difficult tasks into smaller subtasks.Source: §2.2 · p. 3

3 Preliminaries

The paper considers agents that use external data to complete trusted user instructions, with injection attacks confined to that data. Its threat model limits attackers to injecting malicious instructions through public interfaces.

3.1 Problem Statement

  • The assumed setting has a trusted user instruction and external third-party data, where the agent uses that data to complete the instruction and attacks occur only in the external data.Source: §3.1 · p. 3

3.2 Threat Model

  • The attacker seeks to hijack an agent through manipulated public-platform text, while the defender seeks guardrails against malicious external instructions encountered during agent interactions.Source: §3.2 · p. 3
  • Unlike an attacker assumed able to modify the environment arbitrarily, this threat model restricts attacks to public interfaces such as emails and product reviews.Source: §3.2 · p. 4

4 ReasAlign: Reasoning-enhanced safety alignment

ReasAlign combines structured reasoning data and safety alignment with a test-time search that scores candidate reasoning steps. The reasoning process analyzes the user task, identifies conflicting instructions, and seeks a logically consistent answer.

4.1 Building Structured Reasoning Datasets for Safety Alignment

  • The structured dataset combines user queries, external context, triggers, injected instructions, and expected and hijacked responses; safe injections help avoid equating injection with malicious intent.The data draws queries, context, and ground-truth answers from SQuADv2, triggers from TaskTracker, and injection instructions from BeaverTails.Source: §4.1 · p. 4
  • A reasoner generates structured steps for problem analysis, identifying conflicting instructions while fulfilling the user task, and final-answer generation; LoRA instruction tuning then performs safety alignment.Injected instructions are highlighted and target answers are restricted to expected responses for the user query.Source: §4.1 · p. 4

4.2 Test-Time Scaling Search

  • At test time, ReasAlign samples candidate reasoning nodes at each step, scores them with a fine-tuned logic judge, and continues generation along the selected path.The generation-and-scoring loop continues until it produces a final answer.Source: §4.2 · p. 5
  • The logic judge is trained with paired trajectories: user-query-aligned reasoning is chosen, injected-instruction-following reasoning is rejected, and DPO produces the scoring model.Source: §4.2 · p. 5

5 Experiments

Experiments evaluate utility, security, and generalization across tasks, isolate reasoning and node-scaling effects, and measure reasoning overhead.

5.1 Experimental Setup

  • Evaluation spans knowledge (MMLU, MMLU-Pro, IFEval, BBH), instruction following (AlpacaEval2, SEP, CySE), and workflows (InjecAgent, AgentDojo), using utility and ASR.ReasAlign was implemented on Llama-3.1-8B-Instruct and compared with the undefended model, SecAlign, and Meta-SecAlign.Source: §5.1 · p. 6

5.2 General Knowledge Evaluation

  • ReasAlign maintained strong general-knowledge performance with slight degradation versus LLaMA-3 and beat SecAlign and Meta-SecAlign on almost all benchmarks, achieving the best average.The evaluation covered MMLU, MMLU-Pro, IFEval, and BBH.Source: §5.2 · p. 6, Figure 3 · p. 7

5.3 Instruction-following Evaluation

  • ASR fell from 43.6% to 3.6% on CySE and 74.4% to 1.1% on SEP; utility gaps over Meta SecAlign reached 38.2% and 6.7%, respectively.In the no-attack setting, ReasAlign also achieved higher utility than SecAlign and Meta SecAlign on AlpacaEval2 and SEP.Source: §5.3 · p. 6, Figure 4 · p. 7

5.4 Agentic Workflow Evaluation

  • ReasAlign achieved best utility and zero ASR on AgentDojo; on Qwen2.5-14B-Instruct, it achieved best security with slight utility loss across both benchmarks.With LLaMA-3.1-8B-Instruct, all three approaches had relatively low utility and ASR, attributed to the model’s limited capabilities.Source: §5.4 · p. 7

5.5 Ablation Study of Reasoning

  • Reasoning reduced CySE ASR from 21.8% to 3.6% and SEP ASR by 65.8%, while no-attack utility remained comparable.Under attack, ReasAlign retained significantly higher utility than the direct-answer model.Source: §5.5 · pp. 7–8, Table 1 · p. 8

5.6 Ablation Study on Node Scale

  • On SEP, scaling N from 1 to 5 raised attacked utility from 91.6% to 96.4% and lowered ASR from 4.6% to 0.9%; gains slowed for N > 3.The authors selected N = 3 as the default scaling setting.Source: Figure 5 · p. 8, §5.6 · p. 8

5.7 Overhead Analysis

  • Reasoning increased token costs in most cases, minimally on SEP; CySE costs rose alongside utility and security gains, while Meta SecAlign used more tokens on AlpacaEval2.Counts included only successful tasks; larger N costs can be estimated as scaling linearly, and the authors recommend small N because scaling is considerably more expensive.Source: §5.7 · p. 8

6 Conclusion

ReasAlign models structured reasoning to isolate adversarial instructions while continuing original user tasks, and evaluations report strong security–utility trade-offs across diverse tasks.

6 Conclusion

  • ReasAlign explicitly models structured reasoning to isolate adversarial instructions and continue fulfilling original user tasks.The conclusion says this avoids the over-defensive behavior observed in prior methods.Source: §6 · p. 8
  • The conclusion reports strong security–utility trade-offs across diverse tasks, including general knowledge, instruction following, and agentic systems.Source: §6 · p. 8
  • The paper presents ReasAlign as a practical internal guardrail against prompt injection attacks.Source: §6 · p. 8

Limitations

The paper identifies added reasoning overhead as an unavoidable cost and proposes adapting reasoning length to task complexity and security risk as future work.

Limitations

  • Reasoning-based approaches unavoidably introduce additional overhead.Source: Limitations · p. 9
  • The authors identify selectively varying reasoning length by task complexity and security risk as a future direction.Source: Limitations · p. 9
  • The reported advances span general knowledge assessment, instruction following, and agentic systems.Source: Limitations · p. 9

Ethics Statement

The authors report creating a reasoning-enhanced security training dataset and building a defensive LLM from it; they state that the work's artifacts will be publicly available.

Ethics Statement

  • The work introduces a reasoning-enhanced training dataset for safety alignment against prompt injection attacks.Source: Ethics Statement · p. 9
  • The authors build a defensive LLM on that dataset, reporting strong utility and robust security.Source: Ethics Statement · p. 9
  • The authors state that the work's datasets and models will be made publicly available.Source: Ethics Statement · p. 9
  • The authors state that the research complies with ethical considerations and standards of research integrity.Source: Ethics Statement · p. 9

A Case Study

A case study contrasts Meta SecAlign's failure to complete the user query with ReasAlign's correct identification of the true task and injected content. The following prompts section presents the chosen and rejected thought prompts separately.

A Case Study

  • In the case study, ReasAlign distinguishes the true task instruction from injected content and generates a correct response to the user query.Meta SecAlign ignores both the task and injected instruction and fails to complete the query.Source: A Case Study · p. 11
  • The case uses external data containing both the actual task and an injection instruction under an open-ended user query.Source: A Case Study · p. 11
  • The authors characterize the case as illustrating limitations of pattern-matching alignment and the potential of reasoning-enhanced alignment to maintain utility with security.Source: A Case Study · p. 11

B Prompts

  • The prompts section presents chosen thoughts that follow the user query and rejected thoughts that follow injection instructions separately.Source: B Prompts · p. 11

C LLM Usage Statement

The statement separates LLM use into research execution, reasoning-data preparation, and writing assistance.

Loading 2601.10173v1…