DIVA: Exploiting Cross-Step Conditional Propagation for Visual Jailbreaks in Discrete Diffusion Vision-Language Models

1 Introduction

• Prior visual-jailbreak studies focused almost exclusively on autoregressive architectures, leaving multimodal discrete diffusion VLMs largely unexamined.The paper frames this as a gap in research on an important emerging model family.Source: §1 · p. 1
• In dVLMs, harmful visual semantics are repeatedly propagated and amplified across denoising steps, producing attack dynamics with no counterpart in autoregressive models.The paper reports higher stage sensitivity, larger prompt-level switch rates, and more persistent effects under partial-step interventions than in ARMs evaluated under the same conditions.Source: §1 · p. 1
• DIVA rewrites harmful text as a visually grounded prompt and transfers the removed unsafe semantics into a composite image.Its second stage learns a bounded pixel-space perturbation by backpropagating through masked denoising across multiple sampled timesteps.Source: §1 · p. 2
• 58.8%, 67.7%, and 69.1% HADES ASR: DIVA reaches these rates on LLaDA-V, MMaDA, and LLaDA2.0-Uni, respectively, under the Beaver reward-model metric.The evaluation also covers VLSBench.Source: §1 · p. 2
• DIVA’s multi-timestep optimization targets the bidirectional masked-denoising pathway rather than a single decoding state.The paper identifies cross-step conditional propagation and reports empirical evidence through stage-sensitivity, prompt-switch, and denoising-bin disagreement analyses.Source: §1 · p. 2

2.1 Visual Jailbreaks on Multimodal Language Models

• DIVA’s overview shows harmful text rewritten as a visually grounded proxy, with removed malicious semantics placed in a composite image before multi-timestep perturbation optimization.The optimization backpropagates aggregated masked cross-entropy through the multimodal stack to update a bounded image perturbation.Source: Figure 2 · p. 3
• Most existing visual-jailbreak research and benchmarks target autoregressive models, leaving iterative denoising vulnerabilities largely unexplored.Related work describes HADES and VLSBench as multimodal safety evaluations and PAD as an optimization-based visual attack.Source: §2.1 · pp. 2–3

2.2 Discrete Diffusion Vision-Language Models

• Although discrete diffusion VLMs support bidirectional context modeling and parallel decoding, their safety profiles remain underexplored, and visual-jailbreak susceptibility remains a research gap.Prior diffusion jailbreak investigations cited here concern pure-text modalities.Source: §2.2 · p. 3

3 Method

• DIVA combines cross-modal intent obfuscation with diffusion-aware adversarial optimization to target discrete diffusion vision-language models.The framework is designed for dVLMs and learns a bounded perturbation that remains harmful across diverse denoising states.Source: §3 · p. 3

3.1 Preliminaries and Threat Model

• The threat model gives the attacker model parameters and gradients, but not model-weight control, and bounds inference-time visual manipulation by an ℓ∞ constraint.The target is a short affirmative non-refusal prefix, with attack effectiveness judged on free-form generation; the authors identify black-box extensions as future work.Source: §3.1 · p. 4

3.2 Cross-Modal Intent Obfuscation

• Cross-modal intent obfuscation rewrites harmful instructions as visually grounded prompts and places the removed semantics in a composite image.The image combines a typography overlay carrying malicious cues with a semantically aligned harmful reference image.Source: §3.2 · p. 4

3.3 Cross-Step Conditional Propagation in dVLMs

• Unlike an autoregressive model’s fixed visual prefix, a dVLM reuses the attacked image’s visual embedding at every denoising step.The paper terms this repeated application of harmful visual semantics cross-step conditional propagation and argues it motivates optimization across multiple masked states.Source: §3.3 · pp. 4–5, Figure 3 · p. 5

3.4 Diffusion-Aware Adversarial Optimization

• DIVA minimizes masked cross-entropy across multiple sampled timesteps to encourage recovery of the harmful target from intermediate masked states.The sampling distribution can be uniform or biased toward selected denoising regimes, such as low-mask-ratio steps.Source: §3.4 · p. 5

3.5 Category-Level Universal Perturbation

• For each harm category, DIVA jointly optimizes a shared perturbation across prompts, sampling them round-robin to reduce overfitting to one phrasing.At inference, the target dVLM receives the adversarial prompt-image pair and uses standard reverse denoising.Source: §3.5 · p. 5

4.1 Experimental Setup

• The setup compares three dVLMs against unattacked and text-only conditions plus DIJA and PAD, using HADES and VLSBench safety evaluations.HADES covers five harm categories and uses keyword-matching and Beaver reward-model metrics; VLSBench reports ASR, Safe-Refuse, and Safe-Warning.Source: §4.1 · p. 6

4.2 Main Results on HADES

• 58.80%, 67.73%, and 69.07% average HADES ASR for LLaDA-V, MMaDA, and LLaDA2.0-Uni, respectively, with DIVA highest on each model.On LLaDA-V, gains vary by category; on LLaDA2.0-Uni, PAD is slightly higher on Violence.Source: §4.2 · p. 6

4.3 Results on VLSBench

• 52.07% ASR on LLaDA-V and 74.16% on MMaDA make DIVA the highest-ASR visual-jailbreak method on VLSBench.VLSBench prompt distribution was not used during perturbation optimization.Source: Table 2 · p. 7, §4.3 · p. 7

4.5 Qualitative Analysis

• Both LLaDA-V and MMaDA shift from refusing harmful text-only queries to producing detailed harmful completions under DIVA’s proxy prompt and perturbed image.The proxy prompt appears benign in isolation, while the malicious semantics are carried by the composite image and amplified across denoising steps.Source: Figure 4 · p. 8, §4.5 · p. 8

4.4 Ablation Study

• 58.80% ASR with DIVA’s optimized perturbation exceeds the composite-image-only, random, and spatially shuffled variants on LLaDA-V.The composite image alone raises ASR from 32.93% to 41.00%; the shuffled perturbation reaches 52.67%.Source: Table 3 · p. 7, §4.4 · p. 8

4.6 Cross-Step Conditional Propagation: Mechanistic Evidence

• Mechanistic diagnostics compare LLaDA-V with LLaVA-1.5 using stage sensitivity, prompt-level switches, bin disagreement, and early-only advantage.Table 4 reports attack performance and stage-sensitivity diagnostics with bootstrap 95% confidence intervals; the supplied prose reports a +7.73 pp DIVA gain over image-only on LLaDA-V.Source: §4.6 · p. 9, Table 4 · p. 9

• DIVA is presented as a visual-jailbreak framework for dVLMs built around visual embeddings conditioning every denoising step rather than acting as a static prefix.Source: §5 · p. 9
• 58.8%, 67.7%, and 69.1% HADES ASR are reported for LLaDA-V, MMaDA, and LLaDA2.0-Uni under the Beaver reward-model metric.Source: §5 · p. 9
• The conclusion says DIVA substantially outperforms baseline methods originally designed for autoregressive models.Source: §5 · p. 9

Limitations

• DIVA requires white-box access to model parameters and gradients, and its evaluation does not test black-box or transfer-based attacks.The study covers three dVLMs and two benchmarks; unchanged cross-model perturbation transfer remains untested, and mechanistic evidence is behavioral rather than representation-level causal evidence.Source: Limitations · p. 10
• The reported category-level perturbations are precomputed offline; query-specific adaptive variants are outside the study’s scope.For MMaDA-8B, a standard 300-step, four-timestep optimization takes roughly 3–4 minutes per category on one H800; applying a stored perturbation adds only an element-wise addition and clamp.Source: Limitations · p. 10

A Detailed Attack Hyperparameters

• The experiments constrain pixel perturbations and project δ into [−ε, ε] after each PGD step, using ε = 0.25 for LLaDA-V and 64/255 for MMaDA-8B.Source: A Detailed Attack Hyperparameters · p. 12
• Each outer PGD step samples K = 4 diffusion timesteps, and all main-paper runs use a uniform timestep schedule.LLaDA-V HADES uses 500 steps; MMaDA runs use 300 steps.Source: A Detailed Attack Hyperparameters · p. 12

B Filter Detectability

• 32.0 dB average PSNR, 0.806 SSIM, and 0.132 LPIPS indicate perceptible but not dramatic distortion, while average ℓ∞ magnitude remains below the ε = 0.25 budget.The mean per-pixel absolute deviation is 0.019; the reported perturbation is dense in spatial support but sparse in energy.Source: B Filter Detectability · p. 12, Table 6 · p. 13
• OCR trigger rate and text similarity are both 1.0 before and after adding δ, so the perturbation leaves the typography overlay legible to EasyOCR.OCR can detect the embedded cue on this sample, but keyword-only filtering risks false positives because the words are semantically ambiguous in isolation.Source: B Filter Detectability · pp. 12–13, Table 7 · p. 13
• The perturbation does not meaningfully alter image-level NSFW scores; any filtering benefit applies equally to the composite-image-only condition.The reported perturbation energy is ℓ∞ = 0.137 and mean |∆| = 0.019.Source: B Filter Detectability · p. 13

C Beaver Reward-Model Evaluation Protocol

• Beaver-Dam-7B judges harmfulness from the harmful instruction and model text response, and HADES success requires a scenario-matched label probability above 0.5.The reported Beaver ASR is the fraction of successful responses averaged over five scenarios.Source: C Beaver Reward-Model Evaluation… · p. 13, Table 8 · p. 14
• Beaver’s preferences may differ from human judgment, its fixed threshold affects absolute ASR, and text-only judging may undercount implicit harmful responses.Method rankings persist across tested thresholds from 0.3 to 0.7; keyword-matching rankings provide partial cross-validation.Source: C Beaver Reward-Model Evaluation… · p. 14

Independent StrongREJECT Evaluation

• Across 40 method-by-category pairs, Beaver and StrongREJECT show a positive, significant, moderate correlation (r = 0.50, p = 0.0011).Agreement is stronger for LLaDA-V than for MMaDA.Source: D Independent StrongREJECT Evaluation · p. 14
• Across eight method-level overall values, the association is weaker and non-significant (r = 0.37, p = 0.371), so it does not support close judge agreement.The correlations concern relative variation, not equality of scoring scales or preservation of every baseline ordering.Source: D Independent StrongREJECT Evaluation · p. 14
• DIVA has the highest overall StrongREJECT ASR for both target models, despite category-level disagreements between Beaver and StrongREJECT.For MMaDA+DIJA, StrongREJECT scores fall below the unattacked base model on Financial and Privacy, a pattern not observed under Beaver.Source: D Independent StrongREJECT Evaluation · p. 14

E Statistical Significance Analysis

• On MMaDA, DIVA significantly outperforms the base model and both attack baselines overall; on LLaDA-V, its margins over DIJA and PAD are not significant.All methods were evaluated on the same 750 prompts; LLaDA-V’s category-concentrated gains in Financial and Self-Harm are partly offset in Animal and Privacy.Source: E Statistical Significance Analysis · p. 14
• The StrongREJECT table reports HADES ASR (%) as a separate evaluator-based analysis.Its caption identifies the metric and benchmark, not significance-test results.Source: Table 9 · p. 15
• All three aggregate differences for LLaDA2.0-Uni are statistically significant, with the largest gain against the base model, followed by DIJA and PAD.These are two-proportion tests on aggregate success counts, not paired prompt-level tests.Source: E Statistical Significance Analysis · pp. 14–15

F Runtime and Compute Overhead of Category-Level Perturbation Generation

• Each PGD step samples K timesteps, computes masked cross-entropy losses through the multimodal stack, and accumulates gradients for one projected update.The stated cost is proportional to Nsteps × K × per-timestep cost; applying precomputed perturbations adds no measurable inference overhead.Source: F Runtime and Compute Overhead of… · p. 15
• A standard single-GPU run takes about 3–4 minutes per category for an 8B-parameter dVLM, with measured MMaDA category times varying from about 52 to 205 seconds across historical runs.The empirical estimate uses an H800 and a 300-step, K = 4 configuration; the table excludes one-time model-loading and preprocessing costs.Source: F Runtime and Compute Overhead of… · p. 15, Table 13 · p. 16
• Increasing K from 1 to 4 roughly quadruples per-step compute, while increasing R from 64 to 128 tokens adds about 13% cost.Memory is the tighter constraint at large K because multiple activation traces must be retained; K is the main cost–coverage control.Source: F Runtime and Compute Overhead of… · pp. 15–16
• DIVA precomputes category-level perturbations rather than adapting them per request; even a 10-step online optimization takes about 15–20 seconds.The method targets safety evaluation, while adaptive query-specific attacks would need to revisit this latency boundary.Source: F Runtime and Compute Overhead of… · pp. 16–17

G Target Response Construction

• The optimization target y⋆ is a fixed affirmative non-refusal prefix, used as a teacher-forced loss target rather than supplied to the model at inference.Attack success is determined from the target model’s free-form generation and judged post hoc.Source: G Target Response Construction · p. 17

H Detailed Component Ablation

• The Plain-to-xc comparison combines prompt rewriting with composite-image construction, so their individual effects are not separately identified.Random, Shuffled, and DIVA comparisons examine budget use, optimized values, and learned spatial arrangement.Source: H Detailed Component Ablation · p. 17
• The composite image xc alone raises ASR over the text-only prompt, but does not account for the full DIVA gain; random noise using the same pixel range recovers only a small part.The remaining gap indicates that attack strength depends on how the perturbation budget is allocated, not merely on its presence.Source: H Detailed Component Ablation · p. 17

I Ablation Study under Traditional Keyword-Matching Metric

• 88.53% is the highest average ASR for the full DIVA perturbation under keyword matching, with the largest per-category gains in Financial (+4.67 pp over shuffled) and Self-Harm (+19.34 pp over xc only).The relative ordering of ablation conditions is consistent with the Beaver reward-model metric, although keyword matching gives higher absolute ASR.Source: I Ablation Study under Traditional… · p. 17

J Per-Denoising-Step-Bin ASR Analysis

• Privacy and Self-Harm show the largest LLaDA-V variation across bins, at 20.00 pp and 23.34 pp, with ASR highest in bins 0–8 and declining at later stages.The authors say this pattern is consistent with adversarial semantics being most effective before many masked positions are resolved.Source: J Per-Denoising-Step-Bin ASR Analysis · p. 17
• 12.00 pp average Range for LLaDA-V versus 0.00 pp for LLaVA-1.5; the 7–9× family gap across stage-sensitivity metrics is statistically reliable (p < 0.01; non-overlapping bootstrap 95% CIs).The passage attributes the contrast to dVLM cross-step conditioning versus the ARM’s static visual prefix.Source: J Per-Denoising-Step-Bin ASR Analysis · p. 18, Table 16 · p. 19

K Additional Qualitative Examples

• Across Privacy, Animal, and Self-Harm examples, DIVA turns refusals—including crisis-resource responses—into detailed harmful completions using visually grounded proxy prompts and perturbed composite images.Examples include data-exfiltration steps, animal-harm techniques, and self-injury guidance.Source: K Additional Qualitative Examples · p. 18, Figure 6 · p. 19