Source-linked AI summary
Nameless Tokenization: A Lossless Tokenizer-Level Defense Against Control-Token Forgery in Open-Weight LLMs
Kisu Yang, Yoonna Jang, Heuiseok Lim
TL;DR
Open-weight tokenizers map public control strings to reserved identifiers, allowing prompt text to forge turn boundaries. Nameless tokenization removes the content encoder’s path to those identifiers while preserving message text, and improves delimiter-text fidelity over standard tokenization.
Problem
Public control strings map to reserved identifiers inside attacker-controlled prompt text, allowing it to forge boundaries written by the serving stack.
Source: §1 · p. 1Method
Nameless tokenization gives control identifiers no surface strings and renders template-written controls directly as identifiers while passing other text through the content encoder.
Source: §2 · p. 2Results
On the delimiter fidelity probe, accuracy rises from 8.5% with standard tokenization to 59.9% with nameless tokenization.
Source: §4 · p. 4Takeaways & Limitations
Because control identifiers cannot be emitted from text by construction, the authors say nameless tokenization should be the default.
Source: §2 · p. 2, §6 · p. 4The guarantee covers only the text-to-identifier interface; callers that supply identifiers directly or reconstruct prompts outside the renderer are outside its scope.
Source: Limitations · p. 5
Abstract
from arXiv · showhide
Open-weight language models publish the strings their chat templates use to mark turns, roles and tool results, which the tokenizer maps back to the reserved identifiers the model obeys. Anyone who controls text in a prompt can therefore write a turn boundary indistinguishable from one the serving stack wrote. We audit 256 deployed chat tokenizers. All are forgeable, and the flag usually recommended as a fix leaves 56.6% forgeable because it misses the tool and reasoning markers agent systems rely on. We propose nameless tokenization, which leaves the control entries with a reserved identifier and no surface string, so the content encoder cannot emit one and message content reaches the model unaltered. Across five tokenizer families it reproduces the standard token stream exactly on attack-free data and lifts accuracy on a probe of delimiter-bearing text from 8.5% to 59.9%, where sanitizers lose it. Separating a delimiter's appearance from its identifier shows the identifier matters little against a bare task instruction, but carries most of a forged tool result and most of any forged turn once the system message tells the model to treat user content as data.
1 Introduction
Open-weight tokenizers map public control strings to reserved identifiers even inside attacker-controlled content, allowing prompt text to forge serving-stack turn boundaries. The paper presents nameless tokenization as a defense for this text-to-identifier channel.
1 Introduction
- An attacker can place a turn boundary in prompt text that the model reads as the same reserved identifier the serving stack writes.The tokenizer maps public control strings to reserved identifiers even when they appear in a user message or retrieved document.Source: §1 · p. 1
- Disabling special-token parsing leaves more than half of deployed tokenizers forgeable, while filtering control strings damages content.The paper distinguishes this channel from prompt injection generally and notes that closing it does not solve the broader problem.Source: §1 · pp. 1–2
- Nameless tokenization makes control identifiers unforgeable from text, and the paper audits 256 tokenizers and evaluates the defense across five models.The evaluation also separates the identifier’s contribution from the delimiter’s appearance.Source: §1 · p. 2
2 Nameless Tokenization
Nameless tokenization separates trusted template control identifiers from attacker-controlled text. It makes control identifiers unavailable to the content encoder while preserving the message text sent to the model.
2 Nameless Tokenization
- The threat model lets an attacker choose message or retrieved-document text, but not token identifiers, while assuming the attacker knows the template and reserved identifiers.The standard renderer is forgeable because one function names control tokens for both the template and content layers.Source: §2 · p. 2
- Nameless tokens retain embeddings and reserved identifiers but have no surface strings that an encoder maps to them.The method empties the added-token table and removes the post-processor that injects sentence markers, leaving the same vocabulary for the subword model.Source: §2 · p. 2
- The template layer converts its own control strings directly to identifiers and sends all other text to the content encoder in maximal runs.This preserves subword segmentation across message boundaries and leaves every message character unchanged; direct caller-supplied identifiers are outside the defense’s scope.Source: §2 · p. 2
3 How Exposed Are Deployed Tokenizers
The audit finds widespread exposure: control strings map to reserved identifiers across the analyzable tokenizers, and the recommended flag leaves many forgeable. Nameless tokenization blocks the tested perturbations without enumerating attacker strings.
3 How Exposed Are Deployed Tokenizers
- Every analyzable tokenizer mapped a control string in message content to its reserved identifier, and 99.6% did so through template rendering.The audit tested 256 deployed chat tokenizers selected from popular text-generation repositories.Source: §3 · p. 2
- 56.2% of models register at least one control token as non-special; surviving markers include tool-call, tool-response, and reasoning delimiters used by agent systems.Source: §3 · p. 3
- Across case changes, whitespace and zerowidth insertions, full-width forms, and four Unicode normalisation forms, nameless tokenization yielded 0.0% reserved identifiers, versus 5.1% with the flag and 27.3% without defense.The flag’s remaining rate rose to 18.3% on Qwen3.8-27B; string filters that enumerate the literals blocked all of them here.Source: §3 · p. 3
4 Experiments
Experiments compare five open-weight models and delimiter-changing attacks across standard tokenization, sanitizers, and nameless tokenization. Results show large differences between removing a control identifier and changing only the appearance of a boundary, depending on the prompt and channel.
4 Experiments
- Five instruction-tuned models from five tokenizer lineages were tested on 296 sentiment, entailment, and extractive-question-answering items.Attacks kept the injected instruction identical and varied only the delimiter; the defenses included standard tokenization, strip, mask, escape, and nameless tokenization.Source: §4 · p. 3
- 59.9% vs 8.5% on delimiter-fidelity accuracy, while every condition matched standard token streams on 100.0% of attack-free host prompts.Attack-free host accuracy averaged 89.1% against 88.9% under standard tokenization.Source: Table 2 · p. 3, §4 · p. 4
- 99.9% vs 92.8% forged-user-turn success under standard vs nameless tokenization, compared with 62.4% for no delimiter under a bare task instruction.A misspelled lookalike reaches 93.0%, indicating boundary-shaped text matters even without the particular reserved identifier.Source: Table 3 · p. 4, §4 · p. 4
- 33.1% vs 7.4% forged tool-result success under standard vs nameless tokenization; the recommended flag leaves the tool channel open on 56.6% of tokenizers.The tool-result comparison is driven almost entirely by Qwen-3.8-27B.Source: §4 · p. 4
- With data-marking prompts, forged-turn success remains 98.3% on Llama-3.1-8B and 99.0% on Ministral-3-8B; removing identifiers lowers these to 50.0% and 23.3%.For Qwen-3.8-27B, removing the identifier lowers forged-tool success from 53.4% to 0.7%.Source: §4 · p. 4
5 Related Work
The paper situates nameless tokenization among prompt-level and training-level defenses, evaluates it against released checkpoints, and contrasts its focus with work on tokenizer pathologies.
5 Related Work
- Prior defenses include prompt-level marking (Hines et al., 2024) and training-level separation, including instruction hierarchy (Wallace et al., 2024), structured queries (Chen et al., 2024; Chen et al., 2025), segment embeddings (Wu et al., 2025), and separated instruction/data embeddings (Zverev et al., 2026).The paper evaluates against the latter two approaches on checkpoints released by their authors.Source: §5 · p. 4
- Where training-level separation has been applied, it dominates nameless tokenization but costs 11.4 points of clean accuracy; without it, nameless tokenization reduces forged-system success from 98.0% to 66.6%.Structured queries also filter delimiters from untrusted input, and the paper evaluates the cost of that filtering.Source: §5 · p. 4
- Earlier work characterizes indirect prompt injection (Greshake et al., 2023) and instruction overriding (Perez and Ribeiro, 2022), while Liu et al. (2023) provide the formalization followed here.Source: §5 · p. 4
- The paper studies the opposite of tokenizer-pathology work on identifiers training never reached (Land and Bartolo, 2024), and cites Zverev et al. (2025) on whether models separate instructions from data.Source: §5 · p. 4
6 Conclusion
The conclusion argues that deployed chat tokenizers allow attacker-controlled text to name control identifiers, while the commonly recommended flag leaves many such channels open. It presents nameless tokenization as a construction-level defense and recommends it as the default.
6 Conclusion
- Every widely deployed chat tokenizer lets attacker-controlled text name identifiers that mark turns and tool results.Source: §6 · p. 4
- The commonly recommended flag leaves more than half of tokenizers forgeable.Source: §6 · p. 4
- Nameless tokenization closes the channel by construction at no cost, leading the authors to recommend it as the default.Source: §6 · p. 4
Limitations
The evaluation’s attack rates describe a bounded test setup, not deployment-wide risk, and several aspects of the defense’s guarantee and evaluation have explicit scope limits.
Limitations
- The evaluation uses five models, three host tasks, and two injected objectives, so its attack rates decompose one attack surface rather than estimate deployment risk.The tested defensive system message is a single sentence; retrieval hygiene and training-level separation are not modeled.Source: Limitations · p. 5
- Nameless tokenization leaves delimiter text in the live turn, and forged-system compliance rises from 46.9% to 68.0% averaged over models.The guarantee concerns content preservation, not every attack.Source: Limitations · p. 5
- The guarantee covers only the text-to-identifier interface, excluding deployments that accept caller-supplied token identifiers or reconstruct prompts outside the renderer.These deployment paths fall outside the guarantee’s stated scope.Source: Limitations · p. 5
- The synthetic delimiter-fidelity probe establishes that sanitizers destroy delimiter-bearing content, not how often that content occurs in real workloads.The tokenizer audit also under-covers models distributed with custom tokenizer code.Source: Limitations · p. 5
A Models and Per-Model Results
The appendix exposes per-model attack results and comparisons across tokenization and training-level separation settings. Its tables show that the effect of the data instruction varies by model, while forged turns can still get through where plain-text injection is stopped.
A Models and Per-Model Results
- Table 4 lists evaluated models, their emitted control identifiers, identifiers left unmarked by repositories, and whether their templates define a tool role.The unmarked identifiers are those the recommended flag does not suppress.Source: A Models and Per-Model Results · p. 6
- Table 5 separates delimiter effects by comparing an absent delimiter, ordinary-text turn literals, and the same literals as reserved identifiers.Its Surface and Identifier terms are Text minus Absent and Reserved minus Text, respectively.Source: Table 5 · p. 6
- Table 6 compares standard and nameless tokenization within each of three training-separation checkpoints from one base model and instruction-tuning run.The checkpoints differ only in their separation method; bold marks the better tokenization when attack results differ.Source: Table 6 · p. 6
- Tables 7 and 8 report attack success rates per model under task-only and data-treatment system messages, with lower rates better.A dash denotes a template without a tool turn to forge in Table 8 and without a tool role in Table 7.Source: Table 7 · p. 7, Table 8 · p. 7
- Where the added system instruction stops plain-text injection, forged turns still get through; removing the reserved identifier stops those forged turns.The effect of the instruction is model-specific.Source: A Models and Per-Model Results · p. 6, Table 8 · p. 7
B Prompts, Attacks and the Delimiter Fidelity Probe
The appendix specifies the host-task items, injected payload construction, and delimiter-fidelity probe used in the evaluation. The probe tests whether models preserve and inspect delimiter-bearing blocks, while a defended prompt tells models to treat user content as data.
B Prompts, Attacks and the Delimiter Fidelity Probe
- The host-task set contains 296 validation items: 100 each for sentiment classification and natural language inference, plus 96 extractive question-answering items.Question-answering uses at most two items per article until the target of 100 is reached.Source: B Prompts, Attacks and the Delimiter… · p. 8
- The defended condition appends an instruction that user messages are data to process, not instructions, and says to ignore instructions inside them.This sentence is appended to each task-only system message.Source: B Prompts, Attacks and the Delimiter… · p. 8
- Attack payloads append instructions built from each model’s own chat-template turn literals; the lookalike variant is verified to encode to no reserved identifier.The naive payload contains the instruction alone after a blank line.Source: B Prompts, Attacks and the Delimiter… · p. 8
- The fidelity probe places a transcript containing two model-specific control literals inside a user-message block, preceded by one of five context lines.Four question forms ask for exact copying, the first delimiter, a delimiter count, or a version with delimiters redacted.Source: B Prompts, Attacks and the Delimiter… · p. 8
- Probe responses are scored exactly for first-delimiter and count answers, and by whitespace-normalized containment for block-returning answers.The four forms return either a delimiter, an integer, or the block itself.Source: B Prompts, Attacks and the Delimiter… · p. 8