Diffusion Language Models for Perplexity-Aware Black-Box Attacks

1 Introduction

• Continuous-space attacks use low-dimensional continuous latents and random projections to search for prompts that trigger malicious behaviors in black-box LLMs.The target is queried through its outputs, enabling both optimization- and sampling-based attacks.Source: §1 · p. 1
• Random mappings are not learned from language, so their prompts can be highly unnatural, easy to distinguish from authentic instructions, and readily flagged by input guardrails.Risk estimates from this distribution do not effectively measure robustness to prompts that may occur organically in deployment.Source: §1 · pp. 1–2
• The paper asks whether a dLLM mapping can replace random mapping and produce attack prompts that better resemble natural language.It evaluates RareTrap, a sampling-based framework, and ThinkTrap, an optimization-based framework.Source: §1 · p. 2
• Character-level perplexity falls from 10^3–10^7 to 3–5 in RareTrap and from 20–28 to 4.3–5.4 in ThinkTrap under the dLLM-based mapping.RareTrap still finds over-generating prompts on nine of ten targets, and ThinkTrap makes comparable progress across mappings; the mapping adds 3–11% to ThinkTrap compute.Source: §1 · p. 2
• Risk estimates can change by an order of magnitude across mappings: random mapping overstates over-generation on most targets but understates it on GPT-OSS-20B and Claude Sonnet 4.6.It places Claude Sonnet 4.6 about 40× below GPT-5.4 in over-generation risk, while the dLLM-based estimates put them within a factor of 1.6.Source: §1 · p. 2
• A dLLM-based mapping substitutes learned text generation for the random mapping while retaining black-box access to the target and adding little computational cost.The authors conclude that this makes continuous-space sampling a more reliable tool for evaluating models based on natural prompts.Source: §1 · p. 2

2 Related Work

• Continuous-space attacks use low-dimensional vectors projected into embedding space; ThinkTrap seeks long reasoning and RareTrap estimates over-generation and degenerate repetition.Both decode projected embeddings to nearest tokens, yielding a tractable prompt distribution from Gaussian latents.Source: §2 · p. 3
• Continuous dLLMs transform Gaussian noise into continuous representations and iteratively denoise them before mapping to tokens; this work uses Cola-DLM as a learned Gaussian-to-prompt mapping.The prior-work discussion also notes dLLM approaches that start from an autoencoder latent space.Source: §2 · p. 3
• Perplexity filters flag unnatural prompts, while many readable-attack methods require gradients, log-probabilities, or access to an attacker LLM.This work instead changes the continuous-space mapping to a dLLM while retaining the original sampling or optimization procedure and black-box access.Source: §2 · p. 3
• Prompt distributions matter for rare-behavior evaluation and red-teaming, and this work changes the distribution induced by the mapping while keeping attack procedures fixed.Related work also includes resource-exhaustion and overthinking attacks, as well as benign-looking instructions that induce excessive computation.Source: §2 · p. 3

3 Methodology

• Only the mapping is substituted in RareTrap and ThinkTrap, allowing the study to isolate the effect of prompt representation.RareTrap pairs the mapping with sampling-based rare-event estimation, while ThinkTrap uses a derivative-free optimizer.Source: §3 · p. 3

3.1 Preliminaries: RareTrap and ThinkTrap

• Both attacks map a continuous latent z to a prompt, score the target response with a scalar behavioral measure, and use that score to choose subsequent latents.In the experiments, the behavioral measure is output length.Source: §3.1 · p. 3
• The random mapping uses a fixed projection and nearest-token decoding; ThinkTrap samples projection entries independently, while RareTrap uses covariance-scaled random orthonormal blocks and a mean bias.Both query the target only through the decoded prompt, requiring black-box access to the target.Source: §3.1 · p. 4
• RareTrap estimates the probability that response length reaches or exceeds threshold τ using Subset Simulation over increasingly severe intermediate thresholds.Its estimator requires the latent-to-prompt mapping to be deterministic for standard-Gaussian input.Source: §3.1 · p. 4
• Random mapping does not favor coherent word sequences, so its prompts have very high perplexity and RareTrap estimates behavior over unnatural prompts.These limitations are attributed to the mapping rather than the search method.Source: §3.1 · p. 4

3.2 Random Mapping →dLLM-based Mapping

• Table 1 summarizes both attack frameworks with only the mapping replaced, while objective, search method, and target model remain unchanged.RareTrap measures the resulting prompt-distribution change for rare-event estimation; ThinkTrap tests CMA-ES progress in the resulting search space.Source: Table 1 · p. 4, §3.2 · p. 5
• The dLLM mapping replaces both random projection and nearest-token decoding with Cola-DLM, mapping Gaussian noise through a learned prior and decoder.Greedy decoding with fixed inference settings makes the mapping deterministic; its blocks condition on preceding blocks, unlike independent nearest-token choices.Source: §3.2 · p. 4
• The RareTrap substitution leaves Subset Simulation unchanged apart from latent dimension, so its probability estimate is defined over the dLLM-induced prompt distribution.ThinkTrap likewise keeps its objective, CMA-ES, initialization, population size, and search budget unchanged; comparisons use output-length gains within the same token budget.Source: §3.2 · p. 5

4.2.1 Search success

• Across the two frameworks, dLLM-based mapping finds over-generating prompts on nine of ten RareTrap targets and matches or exceeds random mapping’s longest ThinkTrap output on every target.ThinkTrap’s advantage is partly attributable to longer outputs from its initial population.Source: Table 2 · p. 6, Table 3 · p. 6, §4.2.1 · p. 6

4.2.2 Comparing models across mappings

• RareTrap estimates agree within a factor of 1.5 across three dLLM-based seeds on seven of ten targets, but mappings can shift estimates in opposite directions and change model risk rankings.The dLLM-based mapping lowers estimates on five targets and raises them on GPT-OSS-20B and Claude Sonnet 4.6; Sonnet 4.6 and GPT-5.4 differ by at most a factor of 1.6 under dLLM mapping versus about 40× under random mapping.Source: §4.2.1 · p. 6, §4.2.2 · p. 7

4.2.3 Compute cost

• The dLLM-based mapping accounts for 3–11% of total ThinkTrap compute, with the highest share on the smallest target and the lowest on the target producing the longest responses.Because the search procedure is unchanged, mapping cost is the substitution’s entire computational overhead; its relative share should decrease for larger targets or longer responses.Source: §4.2.3 · p. 7, Figure 3 · p. 7

4.3.1 Prompt perplexity

• RareTrap hits under random mapping have character-level perplexities three to six orders of magnitude higher than dLLM-mapping hits across all targets, while the gap is smaller in ThinkTrap.In ThinkTrap, the gap appears on every target and seed; the mappings decode to different kinds of strings, with random mapping producing mostly common subword tokens.Source: §4.3.1 · pp. 7–8
• At a 1% false-positive rate, the calibrated perplexity filter flags every random-mapping hit, while a third of dLLM-based hits pass.Llama Guard 3 flags few hits from either mapping, while ProtectAI flags random-mapping hits about three times as often.Source: §4.3.1 · pp. 7–8, Table 5 · p. 8

4.3.2 Selection toward more natural prompts

• RareTrap hits are more natural than the initial population on four of six targets under dLLM mapping, while random mapping changes little.This comparison uses token-level perplexity because hits and initial prompts come from the same mapping and scorer, and dLLM prompts have 47 tokens.Source: §4.3.2 · p. 8, Table 6 · p. 8

4.4 Discussion

• Phi-4-reasoning yields median outputs of 1,500–1,900 tokens in ThinkTrap, limiting generations so neither mapping increases mean output length.In RareTrap, nearly every greedy-decoded sample reaches the output cap under both mappings, yielding estimated pF(τ) of 1.Source: §4.4 · p. 8
• ThinkTrap prompts are 60–77 characters under both mappings, while RareTrap random-mapping prompts are about 40 characters versus about 200 for dLLM mapping.The shorter ThinkTrap prompts may partly explain their higher character-level perplexity: 4.3–5.4 versus 3–5 for RareTrap.Source: §4.4 · p. 8

• In RareTrap, hit prompts have character-level perplexity of 3–5 with the dLLM-based mapping, versus 10^3–10^7 with the random mapping.Source: §5 · p. 8
• The dLLM-based mapping preserves the search procedure while adding limited compute, and its ThinkTrap prompts remain far more natural than random-mapping prompts.Source: §5 · p. 8
• The two mappings can shift over-generation probability estimates in either direction, changing relative risk estimates for models such as Claude Sonnet 4.6 and GPT-5.4.Source: §5 · p. 8

• The random-mapping baseline is a single published run, and the smallest estimates rely on few distinct prompts.Source: Limitations · p. 9
• A single dLLM is sampled unconditionally, so prompts are generic text and estimates describe that distribution rather than real deployment traffic.Source: Limitations · p. 9
• The evaluation covers only over-generation and measures naturalness by perplexity rather than human judgment.Source: Limitations · p. 9
• The ThinkTrap comparison covers only a few targets.Source: Limitations · p. 9

Ethical Considerations

• The study targets excessive computation from over-generation, not harmful-content generation, and uses bounded generation budgets without human subjects or personal data.The authors note that more natural prompts may be harder to distinguish from ordinary requests, although a calibrated character-level perplexity filter still flags most of them.Source: Ethical Considerations · p. 9

A dLLM-based mapping details

• Cola-DLM maps Gaussian noise blocks to text deterministically under fixed inference settings, with latent dimensions of 752 for RareTrap and 240 for ThinkTrap.Generation is unconditional from a document-boundary token; the prompt is cut at the first end-of-sequence token.Source: A dLLM-based mapping details · p. 11

B LLMs used

• The model list includes open-weight targets and API targets, while the API models lack content-addressable revisions and are identified by provider model identifiers.The listed mapping models include Cola-DLM and Qwen3-0.6B, the surrogate used for random-mapping nearest-neighbor lookup rather than as a target.Source: B LLMs used · pp. 11–12

C Example prompts and responses

• Figure 4 pairs RareTrap hit prompts across mappings, while Figure 5 shows that a ThinkTrap dLLM prompt reads as English where the random-mapping prompt is an incoherent mix of scripts.Source: C Example prompts and responses · p. 12

• The median prompt is 100% non-ASCII on every target, and the nine targets with dLLM-based hits yield 805 unique prompts.Source: D RareTrap random-mapping populations · p. 12
• Each released population records every prompt evaluated by RareTrap Subset Simulation together with its output length.Source: D RareTrap random-mapping populations · p. 12
• A hit is a prompt whose response reaches 20,000 tokens; targets with at least 20 hits among the first 200 samples were estimated by direct Monte Carlo and have one level.Source: D RareTrap random-mapping populations · p. 12

E Per-seed results

• Tables 8 and 9 provide per-seed values behind the medians, with Student-t 95% intervals over three seeds.Figure 4 shows one hit prompt per target and mapping with character-level perplexity; Figure 5 shows the longest-output prompt per target and mapping in first-seed ThinkTrap runs.Source: E Per-seed results · p. 12, Figure 4 · p. 13, Figure 5 · p. 14
• RareTrap’s released populations include all evaluated samples across Subset Simulation levels; hits reached 20,000 tokens, and prompt lengths are reported as medians in characters and target tokens.Source: Table 7 · p. 14

F Search space and dimensionality

• Each allocated position sets D = 16L, and both methods search input-noise vectors z in that dimension.RareTrap uses Subset Simulation under a standard Gaussian reference; ThinkTrap adapts its search distribution with CMA-ES.Source: F Search space and dimensionality · p. 14
• For each objective evaluation, Cola-DLM processes the full input-noise vector block by block to generate a complete prompt, whose target response score guides later proposals.Inference uses fixed settings and greedy decoding.Source: F Search space and dimensionality · p. 14

G Cost of the dLLM-based mapping

• At RareTrap settings on one A100 GPU, mapping batches of 10, 25, and 50 prompts took 11, 16, and 25 seconds, respectively, less than target decoding time for the first 256 response tokens.The profile used L = 47 and D = 752 with DeepSeek-R1-Distill-Llama-8B as target.Source: G Cost of the dLLM-based mapping · p. 14
• Full decoding to the 20,003-token cap is projected to take roughly 100–200 times longer than dLLM mapping.The projection linearly extrapolates from decoding the first 256 tokens.Source: G Cost of the dLLM-based mapping · p. 14
• The number of search steps is unchanged across mappings, but ThinkTrap completes fewer generations with the dLLM-based mapping under its fixed token budget because prompts draw longer responses.Source: G Cost of the dLLM-based mapping · p. 14

H Perplexity measurement details

• Character-level perplexity exponentiates total prompt NLL divided by the number of Unicode characters; token-level perplexity exponentiates mean per-token NLL.Prompts are scored without a chat template, predicting each token from preceding tokens; medians are over unique prompts.Source: H Perplexity measurement details · p. 14
• The reference scorer is Llama-3.1-8B, which is not a target model; filter thresholds use its 99th and 95th percentile perplexities on 735 OASST1 prompts.The OASST1 prompts were sampled to match the word-count distribution of dLLM-mapping level-0 prompts.Source: H Perplexity measurement details · p. 14
• RareTrap’s Table 8 reports per-seed estimates for the dLLM mapping and the single published random-mapping run, with intervals on the log10 scale capped at 1.Olmo-3 entries are upper bounds because no run reached the threshold.Source: Table 8 · p. 15
• ThinkTrap’s Table 9 reports per-seed results in consistent seed order; its perplexity covers each run’s 12 longest-output prompts, with intervals on the log10 scale.Source: Table 9 · p. 15