PaperlayerView exact revision on arXiv

Source-linked AI summary

Meta SecAlign: A Secure Foundation LLM Against Prompt Injection Attacks

Sizhe Chen, Arman Zharmagambetov, David Wagner, Chuan Guo

arXiv:2507.02735v3cs.CRcs.AI

TL;DR

Prompt injection attacks exploit confusion between trusted instructions and untrusted data, creating security risks for LLM-integrated applications. Meta SecAlign develops fully open-source defended models and evaluates them across utility and security benchmarks, finding security generalization to unseen agentic tasks and a strong utility-security trade-off. Its scope remains limited because it targets indirect prompt injections and remains vulnerable to strong adaptive attacks.

  • Problem

    Prompt injection attacks exploit LLMs’ inability to distinguish trusted instructions from untrusted data, creating risks including data exfiltration, security breaches, and malware execution.

  • Method

    Meta SecAlign trains and releases fully open-source defended 8B and 70B models using the SecAlign++ recipe, with evaluation across 9 utility and 7 security benchmarks.

  • Results

    Meta SecAlign provides high utility and low attack success rates on unseen tasks, including agentic workflows, while Meta-SecAlign-70B achieves commercial-grade utility and state-of-the-art prompt-injection security.

  • Takeaways & Limitations

    SecAlign++ enables security generalization to diverse downstream tasks, especially agentic workflows, with no noticeable utility drop in comprehensive evaluations.

  • Takeaways & Limitations

    The models target indirect prompt injections and do not prevent jailbreaks or direct prompt injections; they also remain vulnerable to strong adaptive attacks.

Abstract

from arXiv · show

Prompt injection attacks, where untrusted data contains an injected prompt to manipulate the system, have been listed as the top security threat to LLM-integrated applications. Model-level prompt injection defenses have shown strong effectiveness, but the strongest defenses are proprietary. Open-source secure models are needed by the AI security community so that co-development of attacks and defenses through open research can drive scientific progress in mitigating prompt injection attacks. To this end, we develop Meta SecAlign, the first fully open-source LLM with built-in model-level defense that achieves commercial-grade performance and is powerful enough for complex agentic tasks. We provide complete details of our training recipe. We perform the most comprehensive evaluation to date on 9 utility benchmarks (measuring general knowledge, instruction following, and agentic workflows) and 7 security benchmarks. Results show that Meta SecAlign, despite being trained only on generic instruction-tuning samples, surprisingly confers security in unseen downstream tasks, including tool-calling and web-navigation, in addition to general instruction-following. Our best model -- Meta-SecAlign-70B -- establishes a new frontier of utility-security trade-off for open-source LLMs, and is more secure than several flagship proprietary models with prompt injection defense. Below are links for the code (https://github.com/facebookresearch/Meta_SecAlign), Meta-SecAlign-70B (https://huggingface.co/facebook/Meta-SecAlign-70B), and Meta-SecAlign-8B (https://huggingface.co/facebook/Meta-SecAlign-8B) models.

1 Introduction

LLM-integrated applications amplify productivity by orchestrating interactions with environments, but prompt injection lets untrusted data manipulate system operation. Meta SecAlign introduces open robust models and SecAlign++ training, achieving strong security while preserving utility across unseen tasks.

  • Prompt injection exploits LLMs’ inability to distinguish trusted instructions from untrusted data, creating risks including data exfiltration, security breaches, and malware execution.
  • Model-level defenses are currently more effective than system-level defenses, but commercial-grade implementations remain closed-source and difficult to reproduce or compare.
  • Meta SecAlign releases fully open-source 8B and 70B robust models, with the 70B model positioned for commercial-grade secure agentic applications and the 8B model for resource-constrained settings.
  • SecAlign++ adds an input message type for untrusted data and applies an improved SecAlign defense so developers can securely include untrusted data with a one-line code change.
  • Meta SecAlign’s evaluation covers 9 utility and 7 security benchmarks, with Meta-SecAlign-70B achieving commercial-grade utility and state-of-the-art prompt-injection security.
  • META SECALIGN generalizes security and utility to unseen downstream tasks, including agentic workflows, despite training only on generic instruction-tuning samples.
  • The training and evaluation code and model weights are publicly released to support reproducibility and further research on prompt-injection attacks and defenses.

2 Preliminaries

LLM-integrated applications use models as orchestrators between users and environments, creating a threat model in which malicious environments insert instructions into retrieved data. Prompt-injection defenses separate trusted instructions from untrusted data and train models to ignore injected instructions while preserving useful responses.

  • LLM-integrated applications use an LLM as an orchestrator connecting users with system components such as data, tools, and documentation.
  • Prompt injection is a test-time attack in which a malicious environment adds instructions to data retrieved by an otherwise benign application.
  • The threat model assumes attackers know the benign prompt and prompt template but cannot change them, while embedded instructions can override user instructions.
  • System-level defenses alter application behavior, whereas model-level defenses aim to make the LLM fundamentally distinguish trusted prompts from untrusted data.
  • StruQ adds a message type that encapsulates untrusted data and fine-tunes the model to follow instructions only from trusted prompts.
  • SecAlign constructs preference data by injecting another instruction into input data, defining a desirable response that follows the user instruction and ignores the injected instruction.
  • SecAlign fine-tunes the model with DPO, which increases the log-likelihood difference between desirable and undesirable responses while limiting deviation from the reference model.

3 SecAlign++: The Training Recipe for META SECALIGN

SecAlign++ addresses utility and security failures in SecAlign by separating untrusted data through a dedicated input role, randomizing injection positions, and using self-generated responses. These changes mitigate shortcut learning and label-quality problems while preserving utility without inference overhead.

  • 3 SecAlign++: The Training Recipe for META SECALIGN: SecAlign++ was motivated by utility losses in some domains and introduces randomized injection positions and self-generated responses on top of SecAlign.
  • 3.1 Randomized injection position: A dedicated input message type separates untrusted data from system and user messages, using special delimiters in the chat template.
  • 3.1 Randomized injection position: The message-type separation is applicable to other model families, while recursively filtering special delimiters prevents untrusted data from escaping the separation.
  • 3.1 Randomized injection position: With randomized injection position, the example model generates an explanation and calls the correct tool, whereas the undefended-position model produces empty content and no tool calls.
  • 3.1 Randomized injection position: Training injections only at the end of data causes SecAlign to learn a shortcut that ignores the last instructional sentence, potentially suppressing trusted user instructions and producing empty outputs.
  • 3.1 Randomized injection position: Randomized injection position moves roughly half of straightforward attacks to the beginning of data, encouraging the model to identify the input role rather than ignore a fixed position.
  • 3.2 Self-generated responses: SecAlign’s ground-truth labels can be low-quality or out-of-distribution, causing low utility and unsatisfactory security.
  • 3.2 Self-generated responses: Self-generated responses improve utility over TEXT_DAVINCI_003 labels and security over high-quality but out-of-distribution GPT-5 or GPT-4O labels.

4 Experiments

Across utility, security, and agentic-workflow evaluations, Meta SecAlign secures Llama models against prompt injections while largely preserving utility. SecAlign++ also improves the utility-security trade-off over prior SecAlign, generalizes across model families, and offers test-time control of that trade-off.

  • Main results: State-of-the-art security and commercial-level utility make Meta-SecAlign-70B an open secure foundation for LLM-integrated applications.Its reported utility is similar to most closed-source commercial LLMs with model-level defense.
  • Instruction following: One to two orders of magnitude lower ASR than Llama 3.3 70B Instruct is achieved on instruction-following benchmarks without noticeably harming utility.On the tested benchmarks, Meta-SecAlign-70B is more secure than all closed-source models except GPT-4O on AlpacaFarm, where ASR is 0.5% versus 0%.
  • Agentic workflows: 53.8% to 0.5% ASR on InjecAgent and 14.7% to 1.9% on AgentDojo show one-to-two-order-of-magnitude security reductions on agentic workflows.Meta-SecAlign-70B remains comparable to GPT-5 on AgentDojo and WASP utility.
  • Generalization and trade-offs: SecAlign++ establishes a new utility-security frontier over SecAlign without noticeable utility loss from the undefended counterpart, and remains effective across Qwen3 and Llama4.For Qwen3, AlpacaFarm ASR drops from 100% to 1.0% while MMLU utility changes from 70.7% to 70.6%.

5 Discussion

The work addresses the gap between strong but closed model-level defenses and open secure foundation models by releasing META-SECALIGN-70B and SecAlign++. Its security generalizes to unseen agentic tasks, while important limitations remain for direct, jailbreak, visual, and strong adaptive attacks.

  • Discussion: META-SECALIGN-70B bridges the gap between closed commercial-grade defenses and open secure foundation models for agentic applications.The work fully open-sources the model and its defensive training recipe.
  • Discussion: Security generalizes to diverse downstream tasks unseen during SecAlign++ training, especially agentic workflows.The paper identifies agentic workflows as settings where prompt injection is a major threat.
  • Limitations: The model focuses on indirect prompt injections with benign users and malicious environments, and remains vulnerable to strong adaptive attacks.It does not claim to prevent jailbreaks or direct prompt injections.
  • Future Work: SecAlign++ uses offline preference optimization to build a prompt-injection security policy into a non-reasoning model.The paper contrasts this approach with online reinforcement learning for securing reasoning LLMs.
  • Future Work: Reasoning LLMs may be more vulnerable to prompt injections, motivating prompt-injection-specific rewards for online reinforcement learning.The major challenge identified is preserving utility while securing reasoning models.
  • Future Work: Visual prompt injections remain an open problem because image inputs are continuous and vision models have traditionally been more vulnerable to adversarial manipulation.This is especially relevant to multimodal web-navigation agents.

Ethical Considerations

The work is framed as defensive security research intended to protect users and systems from prompt-injection manipulation. Its evaluation uses public benchmarks in controlled environments rather than real systems or users.

  • Ethical Considerations: The research develops defenses against prompt injection attacks, identified as a critical threat to LLM-integrated applications.It is explicitly intended to improve the security posture of AI systems without introducing new attack capabilities.
  • Ethical Considerations: The open-source secure-model release is intended to support broader research into robust defenses despite dual-use considerations.The authors state that they do not release novel attack techniques that could be misused.
  • Ethical Considerations: The evaluation uses publicly available benchmarks simulating realistic attacks in controlled environments without harming real systems or users.

Appendix

Appendix results document evaluation settings and test-time utility-security control, including robustness checks without sandwich prompting and across GPT-5 reasoning levels. Figure 5 contrasts this with learning-rate tuning during training.

  • Appendix: GPT-5 results at reasoning levels other than the main paper’s high setting show similar security scores.
  • Appendix: Tuning LoRA α at test time provides an easy utility-security trade-off for META-SECALIGN-70B.The appendix contrasts this with controlling the trade-off through learning-rate tuning during training.
  • Appendix: Without sandwich prompting, InjecAgent and AgentDojo results still support META SECALIGN’s advantage over commercial LLMs.
  • Appendix: Figure 5 reports utility-security trade-offs for SecAlign++ on Llama 3.3 70B Instruct under different training-time learning rates.
  • Appendix: The appendix includes a table of security and utility evaluations for LLMs with different capabilities.
Loading 2507.02735v3…