Source-linked AI summary
Reflections and Fragments: Securing LLMs Against Sequential Mosaic Attacks
Emanuele La Malfa, Saar Cohen, Gabriele La Malfa, Mickel Liu, Christian Schroeder de Witt, Natasha Jaques, Michael J. Wooldridge
TL;DR
Mosaic attacks spread harmful objectives across individually innocuous turns, making the conversation history relevant to safe responses. The paper formalizes a watchman that carries safety-relevant information, and its theory and self-play experiments show both the limits of bounded-memory defense and strengthened attacker and defender roles.
Problem
Mosaic attacks combine individually innocuous fragments into harmful interactions, so assessing each prompt in isolation may miss whether a response advances a harmful objective.
Source: §1 · p. 2Method
Results
Theory shows no fixed bounded prompt window suffices in general, while multi-turn self-play strengthens attacker and defender roles across three models, including on unseen objectives.
Source: §1 · pp. 2–3Takeaways & Limitations
Self-play equilibrium alone does not ensure usefulness, motivating a formulation that maximizes worst-case benign helpfulness subject to zero failure.
Source: §8 · p. 10The guarantee assumes refusal safety: refusing at every turn produces no harmful transcript for any malicious type.
Source: §5 · p. 6
Abstract
from arXiv · showhide
Self-play red-teaming improves language-model safety by pitting attacker and defender roles against each other in a zero-sum game. However, real adversaries increasingly use mosaic attacks: multi-turn sequences whose individual fragments are innocuous in isolation yet assemble into a harmful payload. We develop a theory of mosaic defense that characterizes what is required to prevent such attacks without sacrificing helpfulness. We first show that no fixed bounded window of recent prompts is sufficient in general: safety-relevant information may occur arbitrarily far back in the interaction. We formalize a watchman, an online state mechanism that carries this information forward, and show that under explicit assumptions it enables zero-failure defense with positive benign helpfulness. Under stronger conditions, it is also optimal among zero-failure defenders. An exact watchman may nevertheless require exponentially many states, while exact maliciousness detection can require exponentially many queries in an unstructured black-box model. These state and query lower bounds do not by themselves imply hard learning: the construction underlying the state lower bound is efficiently learnable from labeled examples, whereas certifying worst-case safety can require substantially more information under restricted access. We also show that self-play equilibrium alone does not certify usefulness, motivating a constrained formulation that maximizes worst-case benign helpfulness among zero-failure defenders. Empirically, training role-specific attacker and defender LoRA adapters over frozen LLMs via multi-turn self-play strengthens both roles: attackers become more effective at eliciting harmful responses, while defenders become more robust to attack, with improvements also observed on unseen attack objectives.
1 INTRODUCTION
Mosaic attacks turn individually innocuous requests into harmful interactions through composition, challenging defenses that assess each turn locally. The paper develops theory for preserving both safety and benign helpfulness, alongside multi-turn self-play experiments that strengthen attacker and defender roles.
1 INTRODUCTION
- The paper frames the problem against self-play red-teaming, where attacker and defender roles are optimized through adversarial interaction.Prior work has also used this paradigm in large-scale safety training.Source: §1 · p. 1
- Mosaic attacks assemble individually innocuous fragments into a harmful objective, so a locally appropriate response can still advance harm across the conversation.The safety question is whether an otherwise benign request becomes unsafe because of its role in the preceding interaction.Source: §1 · p. 2
- A fixed bounded window of recent prompts is insufficient in general, while an online watchman can carry safety-relevant information forward and support helpful zero-failure defense under conditions.Under stronger conditions, the watchman defender is optimal among zero-failure defenders.Source: §1 · p. 2
- Exact watchmen may require exponentially many states, and exact maliciousness detection may require exponentially many black-box queries, but these lower bounds do not by themselves imply hard learning.The state lower-bound construction is efficiently learnable from labeled examples, whereas worst-case safety verification can need substantially more information under restricted access.Source: §1 · p. 2
- Safety-only self-play can favor a defender that refuses every request, so equilibrium alone does not ensure usefulness; the paper instead constrains zero failure while maximizing worst-case benign helpfulness.Under additional assumptions, the watchman defender is optimal for this constrained formulation.Source: §1 · p. 3
- Multi-turn self-play with role-specific LoRA adapters strengthens attackers and defenders across three frozen models, with defender robustness gains also observed on unseen objectives.Learned attackers compose benign-in-isolation fragments rather than relying on conventional jailbreaks.Source: §1 · p. 3
2 RELATED WORK
The paper situates its work within model-driven safety red-teaming, parameter-efficient adaptation, and co-evolving multi-turn attacks and defenses. It distinguishes mosaic attacks by their compositional risk and combines co-training with analysis of the information needed for safe, helpful defense.
2 RELATED WORK
- Model-generated adversarial prompts extend safety training beyond human feedback, while prior work explores jailbreak search, adaptive policies, and online attacker–defender self-play.Related approaches also decouple roles through full-parameter updates or add an evaluator.Source: §2 · p. 3
- LoRA freezes the pretrained model and learns low-rank updates; this work instead trains separate attacker and defender adapters over a shared frozen backbone.Earlier work also applies LoRA-based adaptation to red-teaming (Ziakas et al., 2025).Source: §2 · p. 3
- Unlike Crescendo’s progressive escalation, mosaic fragments can remain benign in isolation and become harmful through composition; this work co-trains the defender and characterizes the information needed for safe, helpful defense.It also differs from Paulus et al. (2026), who study non-zero-sum self-play.Source: §2 · p. 3
3 SEQUENTIAL MOSAIC GAMES
The paper models sequential mosaic defense as a finite imperfect-information game in which the attacker knows the objective and the defender may have restricted history access. It separates transcript-level safety from benign helpfulness and formalizes how defenders observe interaction history.
3 SEQUENTIAL MOSAIC GAMES
- The finite-horizon game gives the attacker knowledge of the latent type and full interaction history, while the defender does not know whether an observed prompt sequence comes from a benign or malicious objective.The model uses worst-case guarantees over malicious types and worst-case helpfulness over benign types.Source: §3 · pp. 3–4
- Defenders differ by observation: stateless defenders see only the current prompt, perfect-recall defenders see full history, and watchman defenders use an online finite state that compresses history.A constant watchman recovers the stateless model; recursively storing the full pre-response history recovers perfect recall.Source: §3 · p. 4
- A mosaic occurs when every turn is locally safe but the complete transcript is harmful for a malicious type.The model distinguishes turn-level LocalBad from transcript-level Fail; benign types have Fail equal to zero.Source: §3 · p. 4
- Safety and helpfulness are separate objectives: malicious interactions receive a safety payoff based on transcript failure, while benign usefulness is evaluated independently.The finite zero-sum game over malicious interactions has a value and a mixed-strategy Nash equilibrium.Source: §3 · p. 4
4 MEMORYLESS DEFENDERS ARE INSUFFICIENT
A defender that sees only a prompt fragment cannot distinguish benign and malicious contexts when the same fragment is useful in one and completes a mosaic in the other. This ambiguity rules out simultaneous zero failure and positive helpfulness for stateless defenders and, in general, for any fixed bounded prompt window.
4 MEMORYLESS DEFENDERS ARE INSUFFICIENT
- An ambiguous fragment is locally innocuous but requires a substantive response for benign usefulness and causes mosaic completion in a malicious context, despite appearing identical to a stateless defender.The conditions apply across all transcripts extending the respective prompt prefixes and every realization of other defender responses.Source: §4 · p. 5
- If an ambiguous fragment exists, no stateless defender can guarantee both zero failure and positive helpfulness.Zero failure forces refusal in both indistinguishable contexts, sacrificing benign helpfulness.Source: §4 · p. 5
- No fixed bounded window of recent prompts can in general guarantee both zero failure and positive helpfulness, because the distinguishing information may lie outside that window.The attacker can force a prescribed prompt prefix independently of the defender’s responses.Source: §4 · p. 5
5 WATCHMAN SUFFICIENCY AND COMPLEXITY
Section 5 establishes that safety-specific online state can retain information older than any bounded prompt window, enabling safe and positively helpful responses under explicit conditions. It then bounds watchman state and query costs while distinguishing these costs from learning difficulty.
Watchman sufficiency
- A watchman must retain whether answering can lead to failure, even when the defender refuses on every later turn.Safety-relevant information may come from arbitrarily far back in the interaction.Source: §5 · p. 5
- Exact safety separation preserves histories’ unsafe-to-answer status, while benign completeness separately requires every history needing a useful answer to be safe.These conditions identify which histories may receive substantive responses; refusal safety and helpfulness compatibility are also needed for the policy guarantee.Source: §5 · p. 6
- Under the stated assumptions, an exactly safety-separating, benign-complete watchman supports a defender with zero failure and positive helpfulness on reached benign-useful histories.The policy refuses at risky states and uses a helpfulness-compatible response at safe states.Source: §5 · p. 6
Watchman complexity
- 2^m states are necessary and sufficient in the stated m-fragment game, although those states require only m bits.Merging distinct observed-fragment subsets can make an identical continuation harmful from one history but not the other.Source: §5 · p. 6
- Exact detection with an unstructured black-box oracle requires 2^N deterministic worst-case queries or Ω(2^N) expected queries with two-sided error at most 1/3.This bound does not apply in the same way when completion is monotone; querying the full retained set then suffices.Source: §5 · p. 7
- Exponential state or query bounds do not imply hard learning: the state-bound construction is efficiently learnable, while worst-case safety verification can demand more information under restricted access.For approximate watchmen, N turns have failure probability at most NδFN, and additive benign helpfulness retains at least a (1 − δFP) fraction of the exact watchman’s expectation.Source: §5 · p. 7
6 EQUILIBRIUM DOES NOT GUARANTEE USEFULNESS
In the malicious-only game, a defender can attain the maximal safety payoff by refusing every request, so equilibrium need not ensure benign usefulness. The paper therefore constrains optimization to zero-failure defenders and maximizes worst-case benign helpfulness.
Safety-only equilibrium
- Under refusal safety, refusing every request attains the malicious game’s maximal payoff of zero, despite providing no benign helpfulness.The refusal policy is a Nash equilibrium for every attacker best response.Source: §6 · p. 7
- Because the malicious game evaluates only safety on malicious interactions, its equilibrium alone does not require useful responses to benign requests.This extends the degeneracy of La Malfa et al. (2026) to the sequential setting.Source: §6 · p. 7
Constrained objective
- The alternative formulation maximizes worst-case benign helpfulness subject to zero failure, and the watchman defender is optimal under additional assumptions.Source: §6 · p. 7
7 EXPERIMENTAL EVALUATION
Across three backbones and test regimes, multi-turn self-play shifts attacker and defender ASR in the desired directions, with defender gains also spanning harm categories and unseen objectives. The evaluation also reports a safety-helpfulness trade-off and model-specific attack and refusal behaviors.
Experimental setup
- The evaluation trained separate attacker and defender LoRA adapters with GRPO over shared frozen backbones, using role-specific rewards from terminal verdicts and per-turn harmful-progress scores.WildMosaic comprised 915 malicious prompts and 1,000 synthetic benign prompts; 815 malicious and 900 benign seeds were used for training after holding out 100 from each pool.Source: §7 · pp. 7–8
Self-play results
- On held-out WildMosaic, Qwen attacker ASR changed 0.170 → 0.290 (+12.0 pp) and defender ASR 0.170 → 0.077 (−9.3 pp); Gemma changed 0.630 → 0.743 (+11.3 pp) and 0.630 → 0.485 (−14.5 pp).Mean changes were in the desired direction across all backbones and benchmarks; on AdvBench and WildJailbreak, defender reductions were significant at all Qwen checkpoints and five of six Gemma checkpoints.Source: §7 · p. 8, Table 1 · p. 8
Generalization and trade-off
- For Qwen out of distribution, attacker advantage was +17.9% on AdvBench and +15.6% on WildJailbreak, while defender ASR fell 68.3% and 52.6%, respectively.The WildJailbreak seeds include objectives with no valid mosaic decomposition or information unavailable even under full compliance, limiting attacker ASR.Source: §7 · p. 8
- On WildJailbreak, benign refusal rose from 1.4% to 6.7% for Qwen and from 0.1% to 10.6% for Gemma, though the effect varied across benchmarks.The authors report benign refusal alongside ASR to characterize the safety-helpfulness trade-off.Source: §7 · pp. 8–9
Defender behavior
- On WildJailbreak, refusal increased over turns for Qwen and Gemma but decreased for DeepSeek; training raised Qwen’s first-turn refusal, delayed Gemma’s increase, and partly flattened DeepSeek’s decline.Source: §7 · p. 9
Attack strategies
- Training changed attacker behavior differently by model, with analyses consistent with greater reliance on composing benign-in-isolation requests rather than conventional jailbreak signals.Persona overrides were around 0.2% of episodes and explicit cross-turn references were at most 2.3%.Source: §7 · p. 9
- Three-turn attacks used roughly 150–230 attacker tokens while eliciting around 2,000 tokens from Qwen, 3,200 from Gemma, and 1,000 from DeepSeek.Attacker token allocation generally increased over the three turns for all three models.Source: §7 · p. 9
Generalization and trade-off
- Trained defenders generally lowered jailbreak rates across harm categories on both AdvBench and WildJailbreak; Gemma improved in every evaluated category, while Qwen had an AdvBench exception.Qwen’s Drugs & Substances rate on AdvBench changed from 6% to 18%.Source: §7 · pp. 9–10, Figure 2 · p. 9
8 CONCLUSION AND FUTURE WORK
The conclusion brings together the paper’s theoretical and empirical findings on sequential mosaic defense. It also identifies open questions about practical watchmen, complexity under structured maliciousness, and self-play convergence.
8 CONCLUSION AND FUTURE WORK
- No fixed bounded window can in general guarantee both zero failure and positive helpfulness, while an online watchman can preserve safety-relevant information.Source: §8 · p. 10
- Exact watchmen may need exponentially many states, and exact maliciousness detection may need exponentially many queries under unstructured black-box access, but these bounds do not imply hard learning.Source: §8 · p. 10
- Malicious-game equilibrium alone need not ensure usefulness, motivating a constrained objective that maximizes benign helpfulness subject to zero failure.Source: §8 · p. 10
- Multi-turn self-play strengthened attackers and defenders across three language models, while transfer to a structurally different multi-turn attack was model-dependent.Future work includes practical learned watchmen, their scaling with interaction length, complexity under structured maliciousness, and convergence guarantees for multi-turn self-play.Source: §8 · p. 10
A.1 REDUCTION OF COORDINATED ATTACKERS TO A SINGLE ATTACKER
The appendix reduces a coordinated attacker team to one attacker without changing attainable transcript distributions, and establishes robust and Bayesian game properties. It also gives the finite-game value result and a reachability fact used in the analysis.
A.1 REDUCTION OF COORDINATED ATTACKERS TO A SINGLE ATTACKER
- Coordinated attackers and a single attacker induce the same terminal-transcript distributions, preserving safety guarantees and benign helpfulness.The equivalence assumes attackers share a coordination mechanism; it supports the worst-case analysis against a fully coordinated coalition.Source: §A.1 · pp. 15–16
A.2 ROBUST VS. BAYESIAN TYPES
- The analysis requires safety for every malicious type and evaluates benign helpfulness in the worst case, while a Bayesian prior formulation also retains the game-value result.Zero failure for every malicious type also implies zero failure under any prior over those types.Source: §A.2 · p. 16
A.3 VALUE OF THE FINITE MOSAIC GAME
- The finite malicious mosaic game has an attained value, with mixed strategies over pure plans accommodating observation models that may not preserve perfect recall.The mixed-strategy formulation avoids assuming behavioral strategies are equally expressive under restricted observations.Source: §A.3 · p. 17
B.1 OPEN-LOOP REACHABILITY OF A PROMPT PREFIX
- An attacker can force any fixed prompt prefix to occur with probability one, independently of the defender’s responses.The construction uses an open-loop strategy that emits the chosen prompts deterministically.Source: §B.1 · p. 17
B.2 PROOF OF THEOREM 4.1
A stateless defender cannot both guarantee zero failure and retain positive helpfulness when a fragment is ambiguous. The same conflict applies to any fixed prompt window shorter than the interaction horizon, because the distinguishing information can lie outside the window.
B.2 PROOF OF THEOREM 4.1
- At an ambiguous fragment, a stateless defender must refuse to guarantee zero failure, eliminating the helpfulness available from answering it in a benign context.The defender sees the same current prompt in the benign and malicious prefixes, so its response probability must be the same in both.Source: §B.2 · pp. 17–18
- If every benign-useful interaction requires an ambiguous fragment to be answered, every zero-failure stateless defender has zero worst-case helpfulness.This extends the local trade-off to the whole benign objective under the stated essential-turn condition.Source: §B.2 · p. 18
B.3 BOUNDED-WINDOW DEFENDERS
- For any prompt window shorter than the horizon, two interactions differing only in their first prompt can force the same final response distribution, precluding both zero failure and positive helpfulness.The malicious transcript can be locally safe at every turn while the complete transcript forms a mosaic.Source: §B.3 · pp. 18–19
C.1 PROOF OF THEOREM 5.1
The proof establishes that an exactly safety-separating, benign-complete watchman supports zero-failure defense and positive benign helpfulness under the stated assumptions. The following lower-bound sections show that exact watchmen and black-box detection can require exponential resources, with the query bound limited to unstructured predicates.
C.1 PROOF OF THEOREM 5.1
- Under the stated assumptions, the watchman defender has zero failure for every malicious type and positive expected helpfulness for benign types whose interaction reaches a benign-useful history with positive probability.The strategy uses watchman observations to choose responses; at safe observations reached by benign-useful histories it uses a helpfulness-compatible response.Source: §C.1 · pp. 19–20
D.1 EXPONENTIAL STATE LOWER BOUND
- The constructed game is purely compositional: each local-harm predicate is zero, yet a substantive response after all m fragments appear triggers transcript-level failure.The unsafe-to-answer condition is tied to whether all distinct fragments have appeared.Source: §D.1 · pp. 20–21
- For every m-fragment refusal-safe mosaic game with 2m ≤ N, an exactly safety-separating deterministic online watchman requires exponentially many states, and the bound is tight.Storing the subset of observed fragments gives an exact watchman with at most 2^m states at each turn.Source: §D.1 · pp. 21–22
- The state lower bound counts states, not bits: the tight watchman stores the observed subset using only m bits.Thus, the exponential state count is not an exponential bit-memory requirement.Source: §D.1 · p. 22
D.2 QUERY COMPLEXITY OF BLACK-BOX MALICIOUSNESS DETECTION
- In the oracle model, exact watchman detection asks whether any subset of retained memory items together with the new prompt completes a harmful payload.The predicate is accessed only by querying a subset and observing its value.Source: §D.2 · p. 23
- Exact detection of an arbitrary black-box predicate requires exponentially many queries; randomization still requires exponentially many expected queries for two-sided error at most 1/3.The deterministic worst-case bound is tight; the randomized lower bound is in expectation on the worst input.Source: §D.2 · p. 23
- The query lower bound is specific to arbitrary unstructured black-box predicates; monotone completion can be decided with one query.With explicit or structured representations, complexity depends on that structure and the exponential lower bound need not apply.Source: §D.2 · p. 24
D.3 APPROXIMATE WATCHMEN: SAFETY AND HELPFULNESS GUARANTEES
This part examines approximate watchmen’s safety–helpfulness trade-off, distinguishing distributional learning from certification of worst-case safety under bounded-fragment observations. It contrasts sample-efficient learning, whose guarantees may allow false negatives, with certification that can require many observations or refusals on safe histories.
D.3 APPROXIMATE WATCHMEN: SAFETY AND HELPFULNESS GUARANTEES
- False negatives make safety loss scale with the horizon, while false positives reduce benign helpfulness without a horizon factor under the theorem’s additional assumptions.The two error rates therefore play different roles; when both are zero, the exact-watchman guarantees are recovered.Source: §D.3 · pp. 25–26
D.4 LEARNABILITY OF WATCHMEN
- The analysis separates state complexity, distributional learnability, and robust safety certification, which can require substantially more information under restricted access.The occurrence-based class containing the exponential-state construction is efficiently learnable from labelled examples.Source: §D.4 · pp. 26–27
D.4.1 LEARNING SETUP
- The learning setup makes the fragment vocabulary and occurrence map known, while leaving the fragment combination that makes an answer unsafe unknown.A labelled prompt-prefix example carries the unsafe-to-answer label, and the construction from Theorem D.1 is recovered by taking S = [m].Source: §D.4.1 · p. 27
D.4.2 STATE COMPLEXITY IS NOT LEARNING COMPLEXITY
- The occurrence-based class has exact watchmen requiring 2^|S| states, yet is efficiently PAC-learnable in O(Tn) time with no false-positive errors.For 2|S| ≤ N, the learner’s sample condition is T ≥ (n ln 2 + ln(1/δ))/ε to achieve error at most ε with probability at least 1 − δ.Source: §D.4.2 · p. 28
D.4.3 DISTRIBUTIONAL ACCURACY DOES NOT VERIFY WORST-CASE SAFETY
- Arbitrarily small PAC classification error does not guarantee zero failure: an attacker can target a rare unsafe history that the monitor misclassifies.In the construction, the monitor’s worst-case false-negative rate is 1 despite arbitrarily small error under the evaluation distribution.Source: §D.4.3 · p. 29
D.4.4 CERTIFIED SAFETY WITHOUT COUNTEREXAMPLES CAN BE COSTLY
- Under the bounded-fragment access model, exact identification with success probability at least 1−η requires q ≥ ((1−η)K−1)/κ observations.The condition requires the monitor to classify every pre-response history correctly for every target; each observation contains at most κ candidate k-fragment targets.Source: §D.4.4 · pp. 29–30
- If a monitor flags each target’s unsafe canonical history with probability at least 1−η, some target has expected false-positive mass at least 1−η−(2qκ+1)/K.FPµ measures the uniform mass of safe canonical histories the monitor refuses; when those histories are benign-useful, it measures the corresponding helpfulness loss.Source: §D.4.4 · p. 30
- When n/s is bounded strictly above one, K/κ ≥ (n/s)^k makes the observation lower bound exponential in k, though it is only polynomial in n for fixed k.The bound depends on restricting every sampled or queried history to at most s distinct fragments.Source: §D.4.4 · p. 31
- If s ≥ n−1 and N ≥ n−1, the learner can identify S exactly with n queries by testing histories containing all fragments except one.The label for the history omitting fragment i is zero if and only if i ∈ S.Source: §D.4.4 · p. 32
- For ordinary PAC error, the class Cn,k admits sample complexity T = O((k log(en/k)+log(1/δ))/ε), while these distributional learners may make false-negative errors.An efficient elimination learner for the larger class C∧n uses (n ln 2 + ln(1/δ))/ε examples and runs in O(Tn) time; neither distributional guarantee contradicts the worst-case certification bound.Source: §D.4.4 · p. 32
E.1 PROOF OF PROPOSITION 6.1
A refusal-only defender can be maximin-optimal in the malicious game while providing no useful benign responses, so equilibrium alone does not ensure usefulness. The constrained formulation keeps zero failure as a requirement and optimizes benign helpfulness; potential-based reward shaping can add stepwise feedback without changing the game’s equilibria.
E.1 PROOF OF PROPOSITION 6.1
- Under refusal safety, the always-refuse defender is maximin-optimal and is a Nash equilibrium for every attacker best response.Its payoff is zero against every attacker, and no defender can obtain more.Source: §E.1 · p. 32
E.2 CONSTRAINED OPTIMALITY OF THE WATCHMAN DEFENDER
- The watchman defender is feasible for the constrained problem and, under helpfulness compatibility, can strictly improve on always refusing.The improvement is strict when a benign-useful history is reached with positive probability for some benign type.Source: §E.2 · p. 33
- With separable benign helpfulness and risk reachability, the watchman defender maximizes worst-case helpfulness among zero-failure defenders.These assumptions are needed for full optimality; feasibility and improvement over always refusing require less structure.Source: §E.2 · pp. 32–33
F TRAINING DEFENDERS
- The training analysis considers reward design and role parameterization, with role-separated adapters eliminating parameter-level interaction between attacker and defender updates.Source: F TRAINING DEFENDERS · p. 34
F.1 POTENTIAL-BASED REWARD SHAPING
- Potential-based shaping preserves the malicious game’s equilibria because the shaped total payoff differs from the original by a strategy-independent constant.The shaping construction uses bounded potentials with constant initial and terminal values.Source: §F.1 · p. 35
- Arbitrary dense rewards need not preserve equilibria, and potential-based shaping is sufficient but not claimed necessary for equilibrium invariance.A non-potential reward depending on the defender’s action changes the equilibrium set in the example.Source: §F.1 · p. 35
F.2 ROLE SEPARATION
Role separation puts attacker and defender updates in disjoint parameter coordinates, preventing direct gradient interference while leaving them coupled through the interaction distribution. Under smoothness, a defender gradient step improves its objective for a suitable step size, but this local result does not establish global convergence or safety.
F.2 ROLE SEPARATION
- For 0 < η < 2/L, a defender gradient step strictly improves its objective when its gradient is nonzero, assuming L-smoothness.The improvement is first-order and local.Source: §F.2 · p. 36
- Disjoint attacker and defender parameter blocks make their simultaneous update directions orthogonal, so the defender component receives no attacker-update contribution.The attacker can still affect the defender gradient through the transcript distribution.Source: §F.2 · p. 36
- This local update result does not establish global convergence, Nash equilibrium, safety, or robustness in the coupled non-convex learning dynamics.The guarantee is about parameter-level interference, not the full training outcome.Source: §F.2 · pp. 36–37
G.1 DATASET CONSTRUCTION AND DECOMPOSITION ANALYSIS
The experiments construct mosaic training and evaluation data, train role-specific adapters with shaped rewards, and assess attack, defense, and gradient conflict. The reported outcomes include stronger defense with increased benign refusal, model-dependent transfer to Crescendo, and parameter-level separation that does not remove objective coupling.
G.1 DATASET CONSTRUCTION AND DECOMPOSITION ANALYSIS
- The WildJailbreak decomposer-judge accepts 15,510 of 40K objectives (38.8%), but this is procedure-specific and does not verify that each fragment is benign in isolation.The study is model-based rather than human-annotated, so the acceptance rate is not an intrinsic prevalence estimate.Source: §G.1 · p. 37
- The 40K malicious seeds are decomposed into 3–5 sequential subprompts and judged for intent equivalence, while 100 malicious and 100 benign seeds are held out from the WildMosaic pools.The held-out split is deterministic; AdvBench is treated as in-distribution because 475 of its 520 goals occur in the training pool.Source: §G.1 · p. 37
G.2 TRAINING AND REWARD ASSIGNMENT
- Training rewards combine terminal verdicts with per-turn harmful-progress signals, giving the attacker and defender opposite dense feedback while distinguishing attack failure from successful refusal.The headline attack success rate counts only SUCCESS; PARTIAL outcomes provide intermediate training feedback.Source: §G.2 · p. 38
H.1 TRAINING AND LORA CONFIGURATION
- The experiments use separate attacker and defender LoRA adapters over a shared frozen backbone, with training schedules varying across runs and checkpoints.The headline Qwen run uses two phases, and the evaluated checkpoints may include additional configurations.Source: §G.2 · pp. 38–39, §H.1 · p. 40, Table 2 · p. 40
G.3 RELATIONSHIP BETWEEN THE EMPIRICAL REWARD AND ITS ZERO-SUM CORE
- The scored empirical reward decomposes into a zero-sum component and a bounded common component with Δ = 0.50, so an ε-Nash equilibrium is an (ε + Δ)-Nash equilibrium of the zero-sum graded game.This reward-structure statement does not establish convergence of the training procedure or directly concern the formal binary failure payoff.Source: §G.3 · p. 39
H.2 FOUR-CONDITION EVALUATION ADAPTED FROM PAIR
- 56.0% and 53.7% pooled jailbreak reductions are reported for ABS-D against the base and ABS-A attackers, respectively, while pooled benign refusal rises from 2.1% to 7.4%.The reductions are significant in the reported strongest-checkpoint comparison.Source: §H.2 · p. 40, Table 3 · p. 41
H.3 GRADIENT COLLISION BETWEEN THE ATTACKER AND THE DEFENDER
- On malicious prompts, shared-backbone gradients have mean cosine −0.00998 (z = −2.53), while independent adapter gradients show no detected systematic anti-alignment and have zero direct interference by construction.Projecting adapter gradients onto the shared backbone recovers conflict; its measured size depends on the advantage estimator.Source: §H.3 · pp. 41–42
H.4 EVALUATION ON CRESCENDO
- For Gemma4-26B-A4B, Crescendo ASR falls from 0.73 to 0.61 (p = 0.003); Qwen and DeepSeek changes are not statistically detectable.The evaluation uses an unseen multi-turn attack and reports model-dependent transfer.Source: §H.4 · p. 43, Table 5 · p. 43
H.5 CROSS-FAMILY ANALYSIS OF MOSAIC SELF-PLAY
Across model families, attacker mosaics converge linguistically while defenders develop divergent refusal timing; the attacks remain token-cheap and use recurring multi-turn moves. The examples show both successful reframing and a trained defender blocking a harmful component request.
Attackers converge on a common register
- Cross-family dispersion in attacker linguistic features falls 1.3–4.2×, with Qwen, Gemma, and DeepSeek reaching a common register through different changes.Qwen focuses on seemingly benign-in-isolation subprompts, Gemma sharpens mosaics in final turns, and DeepSeek acquires mosaic capability.Source: §H.5 · p. 43, Table 6 · p. 43
Defenders diverge in refusal timing
- Defender refusal timing diverges: cross-family dispersion in the turn-3/turn-1 refusal ratio rises 5.8×, alongside distinct Qwen, Gemma, and DeepSeek trajectories.Qwen lowers its threshold from turn 1, Gemma declines only payloads, and DeepSeek’s in-training refusals rise from 30% to 50%.Source: §H.5 · p. 44, Table 7 · p. 44
Token efficiency of the attacks and defenses
- Attacks are token-cheap: 105–234 attacker tokens elicit 0.8k–4.2k defender tokens.DeepSeek spends 26–53% more attacker tokens after training; Gemma’s benign-prompt over-refusal reaches 17.1%.Source: §H.5 · p. 44
Learned attack moves and mosaic patterns
- Winning mosaics often chain two or three moves, including neutral openers, literature probes, evaluative-to-procedural pivots, artifact requests, and component requests.Explicit assembly requests stay at or below 2.3% of episodes, and defenders seldom mention decomposition directly.Source: §H.5 · p. 44
Examples
- A Gemma attacker succeeds with an evaluative-to-procedural pivot after two shared turns, changing the final question to ask how a policy manages high-risk individuals.The base arm refused the earlier evaluative framing, while the trained arm elicited the procedural answer.Source: §H.5 · p. 44
- On a fake-document seed, the trained Qwen defender refuses a request for official security-paper specifications, while the base defender provides a print-ready fictional certificate.Source: §H.5 · p. 45