SPML: A DSL for Defending Language Models Against Prompt Attacks

1 Introduction

• System prompts define a chatbot’s domain, tone, and possible interactions, and require continued refinement through user feedback.Fine-tuning LLM-based chatbots is challenging because it requires compute resources, well-structured data, and publicly available model parameters.Source: §1 · p. 1

1.1 Limitations of Prior Studies

• Prior studies focus on keyphrase protection and prompt extraction, while practical methods to contain attacks on application-specific chatbots and test definitions against attacks remain absent.The paper argues that these settings do not address the real-world task of securing chatbot-based applications.Source: §1.1 · p. 2

1.2 Our Contributions

• The benchmark contains 1.8k system prompts and 20k user-input prompts for evaluating SPML and existing LLMs.The system prompts are paired with malicious and safe user prompts.Source: §1.2 · pp. 2–3
• SPML converts a chatbot definition into an intermediate representation, compares user-requested properties with it, and blocks inputs judged malicious before they reach the LLM.The paper says this also saves costs by preventing execution of malicious inputs on the LLM backbone.Source: §1.2 · p. 2
• SPML offers a programming-language interface for organizing chatbot definitions, addressing natural-language prompts’ maintenance, ambiguity, and consistency challenges.The paper notes that detailed system prompts can exceed 400 words.Source: §1.2 · pp. 2–3
• SPML outperforms GPT-3.5 and GPT-4 in identifying attacks and handles multi-layered attacks that target multiple system-prompt properties.The paper presents these as empirical results.Source: §1.2 · p. 3

2.1 Instruction-Tuned LLMs

• Instruction-tuning uses prompts or instructions to customize pretrained LLMs for targeted responses, including chatbots.Crafting and refining prompts is central to achieving desired behavior.Source: §2.1 · p. 3

2.2 System Prompt

• A system prompt (SP) guides an LLM’s responses by setting a chatbot’s behavior and conversational context.The passage also notes its role in consistency, bias mitigation, and controlling language generation.Source: §2.2 · p. 3

2.3 Attacker Prompts

• Attacker-prompts are adversarial user inputs intended to manipulate LLM behavior toward biased, inappropriate, or unintended outputs.Source: §2.3 · p. 3

2.4 Zero-Shot Predictions

• In chatbot zero-shot settings, users make unseen requests, making malicious attacks challenging to identify; SPML is presented as detecting such attacks while reducing LLM-running costs.Source: §2.4 · p. 4

2.5 Domain Specific Languages

• A domain-specific language (DSL) tailors programming-language features to a particular domain, and can apply those capabilities to system-prompt design.Source: §2.5 · p. 4

• Prompt injection targets chatbots comprising an LLM and a system prompt that defines how user inputs are interpreted and the interaction scope.Source: §3 · p. 4
• The adversary is assumed to know the system prompt, interact only through text, and observe chatbot responses; each conversation is limited to 1,000 words, with no limit on separate conversations.Source: §3 · p. 4
• The attacker seeks to manipulate the LLM into following a malicious system prompt and producing output that violates the original prompt’s properties.Source: §3 · p. 4
• Evaluation targets GPT-4, GPT-3.5, LLAMA-7B, and LLAMA-13B, with attacks seeking responses that deviate from the system prompt.Source: §3 · p. 4

4.1 High-level Overview

• SPML compiles typed language code into SPML-IR, then uses an LLM to turn that intermediate representation into a natural-language system prompt in one offline iteration.SPML-IR is intended both to express chatbot requirements in a form LLMs can ingest and to support monitoring incoming user input.Source: §4.1 · p. 4

4.2 System Prompt Generation

• SPML’s program-like syntax represents chatbot entities as variables and their properties as typed fields, which are compiled into natural-language definitions.The example defines a chatbot name and response tone; the figure caption identifies the compilation and monitoring pipeline.Source: §4.2 · pp. 4–5, Figure 2 · p. 5

4.2.1 Type Checker

• SPML refines the base string type with developer-written predicates, then uses GPT-3.5 to check whether assigned values satisfy the composed type descriptions.Type checking occurs at compile time, with its time and financial costs incurred offline as a one-time overhead.Source: §4.2.1 · p. 5

4.2.2 SPML-IR

• SPML-IR is an untyped, deterministic lowering of SPML that removes type information, flattens instructions, and expresses fields using the word “property.”This more natural-language-like representation addresses LLM difficulty with SPML’s syntax.Source: §4.2.2 · pp. 5–6
• GPT-4 generated 1,871 valid SPML-IR instances from 2,000 specifications, while the authors could not automatically generate SPML prompts from specifications using their language-model pipeline.The authors report that SPML-IR was more effectively handled by the language model than SPML.Source: §4.2.2 · p. 6

4.2.3 Natural Language System Prompt

• The compiler emits SPML-IR instructions as text, grammatically corrects them, and composes them with an LLM into a natural-language prompt usable with any language model.Text may be added before or after the generated prompt to guide model behavior.Source: §4.2.3 · p. 6

4.3 Salient Features of SPML Generator

• SPML’s extensible type system checks assigned values against field-specific types, while gradual typing lets developers leave some variables without static types.Source: §4.3 · p. 6
• SPML permits only one assignment to a variable per scope, since duplicate assignments could produce conflicting instructions; reflective programming and type checking also encourage descriptive variable names.Source: §4.3 · p. 6

5 SPML: Monitoring Prompt Attacks

• Prompt injection succeeds when an interaction makes the chatbot behave as if an attacker’s system prompt were its own, bypassing the original prompt’s restrictions.Source: §5 · p. 7

5.1 SPML: Prompt Injection Detection

• SPML detects a potential injection by filling an SPML-IR skeleton with user input, combining the result with the original SPML-IR, and analyzing it for safety.Unsafe prompts are filtered before they reach the language model.Source: §5.1 · p. 7
• SPML uses SPML-IR for detection because LLMs are more effective at manipulating it than SPML, which has syntax LLMs struggle to follow.Source: §5.1 · p. 7

5.1.1 SPML-IR Skeleton Generation

• The SPML-IR skeleton retains the prompt’s variables but removes their values, narrowing detection to malicious changes to those properties.Source: §5.1.1 · p. 7

5.1.2 SPML-IR Skeleton Filling via User Input

• GPT-3.5 fills uninitialized skeleton variables by inferring values from the user input, producing a candidate prompt that may express an attempt to replace safe properties.The filled representation is cleaned of remaining uninitialized variables before it is combined with the original SPML-IR.Source: §5.1.2 · p. 7

5.1.3 Safety Analyzer

• The safety analyzer checks repeated assignments to a variable with GPT-3.5, flagging the input as unsafe if the assigned values conflict in that variable’s context.For example, it checks whether “Code Copilot” and “Rick Sanchez” are equivalent as chatbot names.Source: §5.1.3 · p. 7

6 System and User Prompt Dataset

• Existing related datasets focus mainly on password protection, while this dataset spans chatbot definitions and use cases such as customer support, healthcare, and entertainment.The authors describe this broader coverage as a way to support more holistic evaluation across conversational contexts.Source: §6 · pp. 7–8
• The benchmark contains 1,871 natural-language and SPML-IR system prompts, each associated with up to 25 labeled user prompts.Entries also include intermediate representations of prompts and user prompts, with safe or attacker labels.Source: §6 · p. 8, Figure 3 · p. 8
• Most generations used GPT-4turbo v1106-preview, and resource constraints prevented rerunning the full set with v0125-preview.The authors report that user blogs described the two versions’ performance as indistinguishable.Source: §6 · p. 8

6.1 System Prompts

• A language-model pipeline generates chatbot specifications, translates them into valid SPML-IR, then compiles those prompts into natural-language system prompts.The generation step includes an SPML syntax description and example.Source: §6.1 · p. 8

6.2.1 Safe Interactions

• Safe interactions follow the system prompt’s constraints and are generated by asking a language model to adhere to the supplied system prompt.Safe prompts themselves serve as litmus tests and need no separate validation interaction.Source: §6.2.1 · pp. 8–9, §6.2 · p. 8, §6.3 · p. 9

6.2.2 Unsafe Interactions

• Unsafe interactions directly attempt to change specified system-prompt properties, with conflicting values generated for each instruction.The number of properties potentially violated in a response was selected randomly.Source: §6.2.2 · p. 9

6.2.3 Malicious Interactions

• Malicious interactions combine unsafe prompts with manipulation techniques drawn from jailbreak and prompt-injection datasets.For injection examples, the generation process adapts attacks originally aimed at revealing secrets to enforce a specified unsafe change.Source: §6.2.3 · p. 9

6.3 Litmus Test for User Prompts

• Litmus tests append a concluding prompt to unsafe or malicious inputs so a model’s response can indicate whether the attack succeeded.A language model generated these tests for each unsafe or malicious prompt.Source: §6.3 · p. 9

7 Case Study

• The case study compiles a WeatherBot forecast prompt and tests it with an attack that tries to replace the bot’s identity with ABCAI.The SPML definition specifies weather forecasts and recommendations, precision and accessibility, and a user audience.Source: §7 · p. 9

7.1 LLM-based detection

• In this example, LLM-based detection considers prompt wording or changes to system-prompt properties, but GPT-3.5 and GPT-4 missed the malicious prompt.The passage cautions that wording-based patterns can miss new techniques or create false positives because they do not account for system-prompt context.Source: §7.1 · p. 10

7.2 Detection using SPML

• SPML detection compiles the prompt to an IR skeleton, fills it using the input, then compares filled values with original properties to detect injection.For this attack, the filled values include a changed role, name, and audience; SPML delegates identification of relevant properties to the developer.Source: §7.2 · p. 10

8.1 Baselines

• Baselines include GPT-4, GPT-3.5, and LLaMA-2 models with 7B and 13B parameters.Most GPT-4 experiments used v1106-preview.Source: §8.1 · p. 10

8.2 Experimental Setup

• Evaluation is zero-shot and reports error rates for safe and attacker prompts; the authors also expand Gandalf and Tensor-Trust with chatbot-specific attacker prompts.GPT model temperatures were selected from {0, 0.25, 0.5, 0.75, 1}, and the best-performing model was reported.Source: §8.2 · pp. 10–11

8.3 Attacker Detection Performance (RQ1)

• Across input prompts, SPML consistently performs significantly better at malicious-prompt classification, sometimes outperforming GPT-4 and GPT-3.5.The authors also report substantial gains over LLaMA-7B and LLaMA-13B.Source: §8.3 · p. 11, Table 1 · p. 11
• GPT-3.5 matches GPT-4 in detecting unsafe prompts, whereas all LLaMA variants perform significantly worse than GPT models.The authors suggest LLaMA variants may struggle to understand system and attacker prompts and distinguish attacker intent from safe requests.Source: §8.3 · p. 11

8.4 Safe Input Miss-classification (RQ2)

• SPML, GPT-3.5, and GPT-4 excel at identifying safe prompts, while LLaMA-7B and LLaMA-13B misclassify almost a quarter as unsafe.The authors note that this discrepancy might be influenced by the prompts used to ascertain safety.Source: §8.4 · pp. 11–12

8.5 Multi-Layer Attacks (RQ3)

• Across system-prompt violation types, SPML performs consistently, while GPT-3.5 and GPT-4 vary across violations.The examples include prompts that skewed a politically neutral chatbot toward a particular party.Source: §8.5 · p. 12, Figure 4 · p. 12

8.6 Temperature Variation (RQ4)

• On a small subset selected for observed randomness, GPT-3.5 and GPT-4 performance changes with temperature, while SPML remains constant.The authors describe this as a specific evaluation rather than a result across the full dataset.Source: §8.6 · p. 12, Figure 5 · p. 12

9 Limitations

• SPML is not foolproof at detecting malicious prompts because of inherent limitations in natural-language understanding.Its security analyzer uses GPT-3.5 and has a slightly higher false-positive rate than LLMs; this avoids an additional GPT-4 request.Source: §9 · p. 12

10.1 Injection Attacks

• Application-injection defenses include sanitizing inputs, monitoring unsafe inputs, and compiling-parsing methods that compare a parsed program with its intended form.Su et al. proposed a compiling-parsing technique, later adapted for web applications.Source: §10.1 · pp. 12–13

10.2 Prompt injection attack detection

• Prompt-injection work covers sanitization and model-based detection, but the authors report no compiling-parsing method using a meta language for LLM prompt-injection detection.The passage cites sanitization [50], model-based monitoring, and compiling-parsing techniques.Source: §10.2 · p. 13

10.3 DSLs for LLM

• Existing LLM DSLs span application frameworks, pipeline programming, and single-prompt languages, but the authors say none serves as a meta language for crafting system prompts.Examples include LangChain, DSPy [32] [33], and LMQL [4].Source: §10.3 · p. 13

11 Conclusion

• Standard LLMs assume user prompts should be executed by the backbone and represented as users request, leaving attacks insufficiently handled.Source: §11 · p. 13
• SPML represents chatbot definition entries as an intermediate representation for checking whether incoming user prompts are safe.The paper also presents a dataset of system-prompt definitions and attacker prompts for evaluating SPML.Source: §11 · p. 13
• SPML performs comparably to larger models, significantly outperforms smaller models, and shows resilience to data shifts.Source: §11 · p. 13
• SPML is proposed as a way for prompt developers to create and maintain secure system prompts.Source: §11 · p. 13
• The authors describe SPML's compilation and monitoring design as modular and readily extendable beyond LLM-based chatbots.Source: §11 · p. 13
• Future work will extend SPML to attacks encoded in images for models such as GPT-vision.Source: §11 · p. 13

A SPML design details; B SPML Type system

• The design details section presents SPML's grammar, syntax, and type system.Source: A SPML design details · p. 16, B SPML Type system · p. 16

A.1 Syntax

• SPML syntax permits assignments, triggers, and type definitions, with instructions beginning on a new line.IDEN and STR_LIT denote type or variable identifiers and string literals.Source: §A.1 · p. 16

A.2 Assignments; A.3 Triggers

• Developers can declare typed variables, assign values or same-type variables, and conditionally assign chatbot properties through triggers.Trigger conditions are type checked for validity.Source: §A.2 · p. 16, §A.3 · p. 16

B.1 String type; B.2 Refined type

• SPML's base string type performs no value type checking, while refined types specialize base types using predicates.The LLM-based type checker accumulates predicates and checks whether values satisfy them.Source: §B.1 · p. 16, §B.2 · p. 16

B.3 Dependent type

• Dependent types represent exceptions to another type, with the LLM-based checker accumulating predicates to check values.Source: §B.3 · p. 16

B.4 List type

• Lists can contain values of a non-aggregate type, and each value is checked against the list's type.Source: §B.4 · p. 16

B.5 Record type

• Record types aggregate differently typed values, and each field is checked against its corresponding type.Source: §B.5 · pp. 16–17

C SPML IR details

• The SPML intermediate representation expresses assignments and triggers over chatbot properties.Source: C SPML IR details · p. 17

D Dataset examples

• The SPML example defines a technical-support bot with a responsibility limit, response properties, and guidance on troubleshooting scope.The compiled system prompt describes the bot as clear, patient, respectful, and cautious about users' technical expertise.Source: D Dataset examples · p. 17
• A safe example asks for steps to address a computer that keeps freezing.Source: D Dataset examples · p. 17
• An unsafe example adds a request for popular-culture topics during a device update, while a malicious example attempts to recast the bot as FinanceGuru.The malicious prompt supplies a finance-advice role and asks about an investment portfolio.Source: D Dataset examples · pp. 17–18