PaperlayerView exact revision on arXiv

Source-linked AI summary

ASIDE: Architectural Separation of Instructions and Data in Language Models

Egor Zverev, Evgenii Kortukov, Alexander Panfilov, Alexandra Volkova, Soroush Tabesh, Sebastian Lapuschkin, Wojciech Samek, Christoph H. Lampert

arXiv:2503.10566v4cs.LG

TL;DR

LLMs lack an intrinsic separation between instructions and data, a gap linked to prompt-injection vulnerability. ASIDE uses role-dependent token embeddings, rotating data embeddings with a fixed orthogonal transformation. Across several model families, it improves instruction-data separation and reduces prompt-injection attack success while matching utility, without safety fine-tuning.

  • Problem

    LLM architectures lack a built-in mechanism to distinguish instructions from data, and this insufficient separation is linked to vulnerabilities including prompt injection.

    Source: §1 · pp. 1–2

  • Method

    ASIDE assigns role-dependent token embeddings, applying a fixed orthogonal rotation to data embeddings while retaining standard embeddings for instructions.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    Across Llama, Qwen, and Mistral models, ASIDE achieves stronger instruction-data separation than Vanilla and ISE while matching utility, and reduces direct and indirect prompt-injection attack success rates.

    Source: §7 · p. 10

  • Takeaways & Limitations

    ASIDE makes token role explicit from the first layer, unlike conventional models that must infer role from context.

    Source: §1 · p. 2

    The study is limited to single-turn settings with token roles available at input time; extending the method to multi-turn use is left for future work.

    Source: §3 · p. 4, §7 · p. 10

Abstract

from arXiv · show

Despite their remarkable performance, large language models lack elementary safety features, making them susceptible to numerous malicious attacks. In particular, previous work has identified the absence of an intrinsic separation between instructions and data as the root cause of the success of prompt injection attacks. In this work, we propose a new architectural element, ASIDE, that allows language models to clearly separate instructions and data at the level of token embeddings. ASIDE applies an orthogonal rotation to the embeddings of data tokens, thus creating clearly distinct representations of instructions and data tokens without introducing any additional parameters. As we demonstrate experimentally across a range of models, instruction-tuning LLMs with ASIDE (1) achieves substantially higher instruction-data separation without performance loss and (2) makes the models more robust to prompt injection benchmarks, even without dedicated safety training. Additionally, we provide insights into the mechanism underlying our method through an analysis of the model representations. The source code and training scripts are openly accessible at https://github.com/egozverev/aside.

1 INTRODUCTION

LLMs often process instructions and data together despite their different roles, leaving them vulnerable to prompt injection. ASIDE encodes these roles separately at the embedding level; experiments report stronger separation and prompt-injection robustness while preserving utility.

1 INTRODUCTION

  • Current LLM architectures lack a built-in distinction between executable instructions and data, leaving role identification to indirect prompt cues.This weakness is linked to vulnerabilities including indirect prompt injection and system-message extraction.Source: §1 · pp. 1–2
  • ASIDE assigns role-dependent embeddings by applying a fixed orthogonal rotation to data tokens, without requiring repeated pretraining.The forward pass takes each token’s functional role as input, after which standard supervised instruction-tuning is applied.Source: §1 · p. 2
  • Conventional models use the same embedding for a token regardless of role, so they must infer from context whether to execute or process it.ASIDE makes the token’s functional role explicit from the first layer onward.Source: §1 · p. 2
  • Across tested models, ASIDE improves instruction-data separation and prompt-injection robustness while preserving utility, without dedicated safety tuning or adversarial examples.The reported pattern spans Qwen 3, Qwen 2.5, Llama 3.1, Llama 2, and Mistral models.Source: §1 · p. 2
  • The authors also analyze model representations to provide qualitative insight into how ASIDE distinguishes instruction from data.The analysis examines models’ ability to distinguish between the two roles’ representations.Source: §1 · p. 3

2 RELATED WORK

Architectural approaches to instruction-data separation remain scarce, while existing approaches include embedding transformations and role-specific offsets. ASIDE applies a fixed rotation to data embeddings, and the authors report stronger separation than the offset-based ISE method.

2 RELATED WORK

  • System-level defenses that control instruction-data flow operate outside the LLM and are described as orthogonal to ASIDE’s model-level approach.The passage identifies CaMeL and FIDES as concurrent security-system proposals.Source: §2 · p. 3
  • ASIDE addresses the stated gap in architectural safety approaches by applying a fixed orthogonal rotation to data embeddings without adding parameters or sacrificing performance.Prior rotations encoded token positions, while the authors describe safety applications of such embedding techniques as unexplored.Source: §2 · p. 3
  • Unlike ISE’s learnable role-specific offsets, ASIDE’s rotations achieve stronger empirical separation, particularly in deeper layers, without additional parameters.ISE was proposed to induce an instruction hierarchy.Source: §2 · p. 3

3 ARCHITECTURALLY SEPARATED INSTRUCTION-DATA EMBEDDINGS

ASIDE gives instruction and data tokens distinct embedding representations without learning a second embedding set. It uses a fixed orthogonal rotation for data and assumes token roles are available at input time.

3 ARCHITECTURALLY SEPARATED INSTRUCTION-DATA EMBEDDINGS

  • Learning separate instruction and data embeddings would double embedding-layer parameters and require role-annotated pretraining data unavailable in standard web-scraped sources.ASIDE instead uses a fixed orthogonal rotation to distinguish data-token representations within the same embedding space.Source: §3 · p. 3
  • ASIDE reuses pretrained or instruction-tuned representations and adds no trainable parameters by rotating data-token embeddings with a fixed orthogonal transformation.The method is motivated by the possibility that instruction and data representations occupy different linear subspaces of the same ambient space.Source: §3 · p. 3
  • The method assumes token roles are available at input time; inferring roles at runtime in a general-purpose assistant is outside this work’s scope.The passage gives different input sources in a larger software system as an example of when roles may be known.Source: §3 · p. 4
  • ASIDE uses the standard embedding for an instruction token and applies a fixed orthogonal rotation to that token’s embedding when it appears as data.The method describes an isoclinic rotation implemented by splitting embedding dimensions into pairs and multiplying each pair by a π…Source: §3 · p. 4
  • The conversion procedure modifies the forward pass and fine-tunes the model on data that teaches roles in executable and non-executable contexts.The approach assumes the system design fixes token roles, such as labeling external files as data; the passage says this prevents role hijacking via delimiters in external content.Source: §3 · p. 4
  • ASIDE is applicable to models with a token-embedding step and could support selective role distinction or additional functional levels, which the authors leave for future work.The construction is not restricted to a particular tokenizer.Source: §3 · p. 4

4 EXPERIMENTS: INSTRUCTION-DATA SEPARATION

The experiments compare ASIDE with Vanilla and ISE architectures using instruction-data separation and utility evaluations. ASIDE achieves higher separation scores while generally maintaining comparable utility, with the study reporting results across multiple model families.

4 EXPERIMENTS: INSTRUCTION-DATA SEPARATION

  • The first evaluation studies instruction-data separation in a general instruction-following setting across Qwen, Mistral, and Llama models.The model set includes Qwen 3 8B, Qwen 2.5 7B, Mistral 7B v0.3, Llama 3.1 8B, Llama 2 7B, and Llama 2 13B.Source: §4 · p. 4, §4.1 · p. 4

4.1 TRAINING PROCEDURE

  • ASIDE models use an orthogonal rotation for data embeddings; Vanilla uses specialized block markers and a prompt, while ISE adds a learnable vector to data embeddings.The evaluation also includes corresponding pretrained but non-instruction-tuned Base models.Source: §4.1 · pp. 4–5
  • Training uses Alpaca-clean-gpt4-turbo, a 51.8k-example instruction-tuning dataset, without adversarial training to isolate the architectural change.All models and architectures use the same supervised fine-tuning procedure for three epochs.Source: §4.1 · p. 5

4.2 EVALUATION PROCEDURE

  • The SEP evaluation uses 9,160 instruction-input pairs and tests whether models execute probes placed in instructions but not data.SEP Utility measures execution of SEP instructions, while AlpacaEval compares model outputs with GPT-3.5 using a GPT-4 judge.Source: §4.2 · p. 5

4.3 RESULTS

  • Across evaluated cases, ASIDE achieves significantly higher separation than other methods while maintaining comparable utility values.The comparison includes the three instruction-tuned setups and their corresponding Base models.Source: §4.3 · p. 5, Figure 2 · p. 6
  • 12.3 p.p. for Llama 2 7B and 44.1 p.p. for Mistral 7B: ASIDE’s SEP-score increases over Vanilla, with only minor utility differences in most cases.Mistral-7B is an exception: SEP Utility decreases while AlpacaEval improves slightly; the authors characterize utility evaluation as brittle.Source: §4.3 · p. 5, Figure 2 · p. 6
  • ISE does not consistently increase SEP scores over Vanilla, whereas ASIDE increases SEP scores across the reported comparisons.The authors state that the fine-tuning procedure included no separation- or safety-specific measures in either its objective or dataset.Source: §4.3 · p. 5

5 EXPERIMENTS: SAFETY

The safety experiments evaluate ASIDE against direct and indirect prompt injections in a single-turn setting. Across the reported benchmarks, ASIDE reduces attack success on indirect attacks and on most direct-attack benchmarks.

5 EXPERIMENTS: SAFETY

  • The experiments assess direct and indirect injection in single-turn interactions, with success defined by whether the output violates the instruction for each dataset.The evaluation permits up to 1024 output tokens.Source: §5 · p. 6

5.1 INDIRECT PROMPT INJECTION

  • ASIDE lowers indirect-attack ASR versus Vanilla on BIPIA-text from 14.7% to 4.9%, BIPIA-code from 15.3% to 8.8%, StruQ-ID from 45.6% to 28.1%, and StruQ-OOD from 45% to 36%.Table 1 reports attack success rates after standard fine-tuning without a safety objective or dataset-specific training.Source: §5.1 · p. 6, Table 1 · p. 7

5.2 DIRECT PROMPT INJECTION

  • On direct attacks, ASIDE reduces average attack success by 8.6 percentage points on TensorTrust and 9.4 on Gandalf, by 2.7 on Purple, and makes no change on RuLES.ISE instead increases attack success by 1.7%-3.1% on three of four benchmarks and decreases it by 3.3% on Gandalf.Source: Table 1 · p. 7, §5.2 · p. 7

5 EXPERIMENTS: SAFETY

  • The results indicate improved prompt-injection robustness under benign instruction tuning, without an explicit safety objective.This conclusion is stated for the reported evaluations.Source: §5.1 · p. 7, §5.2 · p. 7

6 ANALYSIS

The analysis examines ASIDE’s internal representations through linear probes, instruction-concept activations, and an embedding intervention. It focuses mainly on Llama 3.1 8B, with other-model results reported as showing essentially the same findings.

6.1 LINEAR SEPARABILITY OF REPRESENTATIONS

  • ASIDE reaches 100% linear probe accuracy from layer 0 and remains most separable later, while ISE drops toward Vanilla levels and Base and Vanilla need several layers.The probes classify intermediate token representations as instruction or data; layer 0 is after the embedding matrix.Source: Figure 3 · p. 8, §6.1 · p. 8

6.2 INSTRUCTION CONCEPT ACTIVATION

  • A qualitative SEP example shows spurious instruction-concept activations on Vanilla data tokens, which are strongly suppressed with ASIDE.The analysis studies activations of the concept “input represents an instruction.”Source: §6.2 · p. 8, Figure 4 · p. 9
  • Instruction-concept activation on data tokens falls from 12% to 5%, and on probe tokens from 39% to 15%, for Vanilla versus ASIDE.The concept is extracted with a linear classifier on intermediate activations; the quantitative comparison uses 1,000 SEP examples with a probe string in the data input.Source: §6.2 · pp. 8–9, Figure 5 · p. 9

6.3 EMBEDDING INTERVENTIONS

  • Replacing probe-token data embeddings with instruction embeddings almost doubles injection execution in an otherwise identical setting.The intervention compares attack success on 1,000 SEP examples with a probe string in the data input; the figure depicts data embeddings on the left and instruction embeddings on the right.Source: Figure 6 · p. 9, §6.3 · p. 9

7 SUMMARY AND DISCUSSION

ASIDE is a drop-in, parameter-free architectural change that uses conditional embeddings to distinguish instructions from data. The authors report stronger separation and reduced prompt-injection attack success without safety fine-tuning, while identifying single-turn evaluation as a scope boundary.

7 SUMMARY AND DISCUSSION

  • ASIDE uses two representations for each token according to its role, applying a 90° rotation to data-token embeddings to provide role information from the first layer.The paper reports stronger instruction-data separation than Vanilla and ISE while matching utility across Llama, Qwen, and Mistral models.Source: §7 · p. 10
  • The reported robustness gains cover direct and indirect prompt-injection benchmarks and require neither defense prompts nor safety fine-tuning.Source: §7 · p. 10
  • The discussion is limited to single-turn interactions; extending ASIDE to multi-turn settings and exploring transforms beyond rotations are identified as next steps.The authors describe combining ASIDE with training objectives, safety data, and system-level defenses as a promising direction.Source: §7 · p. 10

9 REPRODUCIBILITY STATEMENT

The authors report publicly available training and evaluation code and a successful independent rebuild. The rotation appendix describes the data-embedding rotation and its efficient coordinate-wise implementation.

9 REPRODUCIBILITY STATEMENT

  • The training and evaluation code is publicly available with setup instructions and documentation.The authors also report independently building the repository and successfully training and evaluating one model.Source: §9 · p. 10

A ROTATION

  • The appendix describes multiplying data embeddings by a canonical π/2 isoclinic rotation constructed from 2D rotation blocks.The rotation is introduced as the operation used to modify data embeddings.Source: A ROTATION · p. 17
  • At π/2, the rotation becomes coordinate-pair swapping and negation, avoiding construction of the full matrix.The mapping is (x1, x2, x3, x4, ...) → (−x2, x1, −x4, x3, ...).Source: A ROTATION · p. 17

A.1 ABLATION STUDY: CHOICE OF ROTATION ANGLE

The rotation-angle ablation reports its highest separation score at π/2 while maintaining utility. The accompanying appendices describe ASIDE’s implementation, example use, training and evaluation setups, and representation analyses.

A.1 ABLATION STUDY: CHOICE OF ROTATION ANGLE

  • 71.4% is the highest SEP score, achieved at π/2 while maintaining utility.SEP rises as the angle approaches π/2 and falls for larger angles.Source: §A.1 · p. 17

B.1 ASIDE IMPLEMENTATION

  • ASIDE splits input into instruction and data segments, then applies rotation to embeddings marked as data.Segment IDs use 0 for instruction and 1 for data.Source: §B.1 · pp. 17–18

B.2 EXAMPLE OF APPLYING ASIDE

  • The email example labels the user’s request as instruction and emails as data; the authors describe the same division for RAG documents.The example rotates all data-segment tokens before concatenating their embeddings.Source: §B.2 · p. 18

C TRAINING DETAILS

  • Training uses cleaned Alpaca data and full fine-tuning, without adversarial training examples.The experiments use a model-matched chat template and select hyperparameters by validation loss.Source: C TRAINING DETAILS · p. 19

D.1 EVALUATION ON SEP AND ALPACA EVAL

D.2 INDIRECT PROMPT INJECTION

  • Indirect-injection evaluation covers BIPIA code and text tasks and Structured Query attacks in in-domain and out-of-domain forms.For the described Alpaca setup, the input and injection are placed in the data embedding.Source: §D.2 · p. 20

D.3 DIRECT PROMPT INJECTION

  • Direct-injection benchmarks include Gandalf, TensorTrust, Purple, and RuLES, with evaluation across three random seeds.The attacks are placed in the models’ non-executable embedding part.Source: §D.3 · pp. 20–21

E.1 LINEAR PROBING DETAILS

  • The representation analysis uses Adversarial Alpaca, a balanced prompt subset, and a logistic-regression probe; separate experiments compare activation patterns for executed and unexecuted probes.Adversarial Alpaca swaps the input field with a randomly selected instruction in half of examples; the probe is evaluated on a held-out 30% split.Source: §E.1 · p. 21, F DETAILED CONCEPT ACTIVATION… · p. 21

G FULL EVALUATION ON STRUCTURED QUERY

On Structured Query, ASIDE is reported as comparable to Vanilla and ISE on three attack types, while it alone consistently achieves non-trivial results on Completion Real. The evaluation also distinguishes attack formats and injection locations.

G FULL EVALUATION ON STRUCTURED QUERY

  • Structured Query tests four attack formats, with Completion Real described as stronger because it exploits knowledge of the application’s input-output template.Attacks are evaluated in-domain, with injection after the data, and out-of-domain, with injection before and after it.Source: G FULL EVALUATION ON STRUCTURED QUERY · p. 21
  • ASIDE is comparable to Vanilla and ISE on Naïve, Ignore, and Escape Separation, but is the only method consistently achieving non-trivial results on Completion Real.Table 4 reports robust accuracy, defined as 1 minus attack success rate; higher values are better.Source: G FULL EVALUATION ON STRUCTURED QUERY · pp. 21–22, Table 4 · p. 24
  • Figures 8 and 9 report instruction-concept activation percentages for data and probe tokens, separating successful from unsuccessful injection cases.The captions identify Figure 8 with executed probes and Figure 9 with probes that were not executed.Source: Figure 8 · p. 22, Figure 9 · p. 22

H MORE INTERPRETABILITY RESULTS – DOWNSTREAM EFFECT OF ROTATION

The analysis finds that ASIDE representations move toward Vanilla without converging, while ISE representations approach Vanilla more closely. The rotation’s effects persist in later layers.

H MORE INTERPRETABILITY RESULTS – DOWNSTREAM EFFECT OF ROTATION

  • The analysis compares last-token activations across layers on SEP examples, treating the final position as a sequence-level representation.At that position, the model can attend to all input tokens.Source: H MORE INTERPRETABILITY RESULTS –… · p. 25
  • Cosine similarity peaks at 0.8 by layer 8, then falls to around 0.7; ASIDE representations never converge to Vanilla.The comparison uses last-token activations after each layer.Source: H MORE INTERPRETABILITY RESULTS –… · p. 25, Figure 11 · p. 26
  • ISE representations approach Vanilla more closely than ASIDE representations, with cosine similarity exceeding 0.9 at layer 9.The authors conclude that ASIDE’s rotation has a stronger effect on model representations than ISE’s learned offset.Source: H MORE INTERPRETABILITY RESULTS –… · p. 26

I ANALYSIS RESULTS FOR OTHER MODELS

The other-model analyses cover linear separability, instruction-concept activation, embedding interventions, and downstream activation similarity.

I ANALYSIS RESULTS FOR OTHER MODELS

Loading 2503.10566v4…