PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety

Fengpeng Li, Kemou Li, Qizhou Wang, Haiwei Wu, Jiantao Zhou, Di Wang

arXiv:2610.01323v1cs.AI

TL;DR

Single-turn preference objectives do not by themselves bound risk over unseen multi-turn histories. TRACE applies token-level preference supervision with continuation-weighted attribution and localized erasure; it reports the lowest attack success rate across all evaluated model–attack pairs. Its risk analysis is conditional on coverage and transfer assumptions and evaluates the base policy on contexts visited by the trained policy.

  • Problem

    Response-level preference objectives score complete responses under fixed prompts, leaving no nontrivial distribution-free bound on multi-turn trajectory risk without coverage assumptions.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    TRACE weights preferred-response tokens by discounted refusal-attributable evidence, erases selected rejected-response positions using policy alternatives, and regularizes with a gradient-norm penalty.

    Source: §1 · p. 2, §4.1 · pp. 4–5

  • Results

    TRACE achieved the lowest ASR in all 35 model–attack pairs, lowering mean multi-turn ASR from 52.81% to 28.98%, while MMLU and HellaSwag dropped by at most 1.23 points.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The analysis connects local suppression at supervised contexts to a trajectory-risk budget when coverage and transfer conditions hold, while accounting for leakage.

    Source: §3 · p. 4

    The contraction bound compares against the base policy evaluated on contexts visited by the trained policy, not against the base policy’s own trajectory risk.

    Source: §3 · p. 4

Abstract

from arXiv · show

Safety-aligned large language models (LLMs) often refuse a harmful request but comply once the same goal is spread over several turns. Preference objectives score whole responses to single prompts, so their training loss alone cannot control risk on unseen histories. Our analysis gives sufficient conditions under which suppression at supervised single-turn contexts yields a bound on multi-turn trajectory risk. The bound accounts for coverage, transfer slack, and leakage, and characterizes contraction relative to a base-policy risk budget evaluated on the trained policy's contexts. TRACE (Trajectory Return Attribution and Contrastive Erasure) turns this principle into a token-level objective. On the safe response, each token is weighted by the discounted return of a refusal-attributable advantage. The advantage compares a frozen reference model with its refusal-ablated copy, allowing earlier response tokens to receive credit from later refusal-related evidence. At high-gap positions on rejected responses, TRACE combines the observed token with policy-selected alternatives in the erasure target. A gradient-norm penalty replaces the retain set. Across five open-weight models and seven multi-turn attacks, TRACE gives the lowest attack success rate (ASR) in all 35 model and attack pairs, while the model utility evaluated on MMLU and HellaSwag drop by at most 1\.23 points. Source code can be found in the supplemental material.

1 INTRODUCTION

The paper asks how single-turn supervision can improve safety across multi-turn histories that differ from training contexts. TRACE allocates preference supervision at the token level, and the paper reports lower multi-turn attack success across its evaluated models and attacks.

1 INTRODUCTION

  • Multi-turn adversaries can gradually elicit harmful assistance through histories shaped by user queries and the model’s own replies.The paper calls failures of safety across interaction histories trajectory fragility.Source: §1 · p. 1
  • TRACE combines discounted refusal-attributable weighting on preferred responses with localized proxy erasure on rejected responses and a gradient-norm penalty.The penalty uses the same preference batch, without an additional retain dataset.Source: §1 · p. 2
  • TRACE achieved the lowest attack success rate in all 35 model–attack pairs, lowering mean multi-turn ASR from 52.81% to 28.98%.MMLU and HellaSwag dropped by at most 1.23 points.Source: §1 · p. 2
  • The paper derives a conditional trajectory-risk bound linking local suppression to coverage, transfer slack, and leakage.It also shows discounted returns can rank an earlier token above a later token with greater immediate advantage.Source: §1 · p. 2

2 PRELIMINARIES

Preference objectives score responses under fixed prompts, whereas deployment safety depends on interaction histories shaped by both user queries and model replies. The paper frames multi-turn safety in terms of unsafe responses accumulated along those trajectories.

2 PRELIMINARIES

  • Preference optimization raises a preferred response’s reference-normalized likelihood relative to a rejected response, scoring complete responses under a fixed prompt.Source: §2 · p. 2
  • At deployment, each response is generated from a history containing prior user queries and model replies, so the history distribution can differ from static preference data.Attacks can shape context gradually before an unsafe continuation becomes likely.Source: §2 · p. 2
  • Trajectory risk captures unsafe responses accumulated across turns, so direct single-turn safety can coexist with vulnerability over interaction histories.Unsafe responses are defined relative to the current context.Source: §2 · p. 3

3 TRAJECTORY FRAGILITY AND THE SINGLE-TURN TRANSFER PRINCIPLE

The section shows that static response-level preference loss alone cannot bound multi-turn trajectory risk. It gives conditions under which suppression at supervised single-turn contexts contracts a risk budget evaluated on contexts visited by the trained policy.

3 TRAJECTORY FRAGILITY AND THE SINGLE-TURN TRANSFER PRINCIPLE

  • Without coverage assumptions, static preference loss gives no nontrivial distribution-free upper bound on trajectory risk, even with large preference margins.Policies can have identical preference losses but trajectory risks differing by T; a favourable margin also need not imply safety at every critical token context.Source: §3 · p. 3
  • The analysis defines local unsafe mass at token contexts and weights coverage by base unsafe mass under the new policy’s visitation law.This lets high-risk contexts dominate the budget and makes leakage relevant to the net effect of covered suppression.Source: §3 · p. 3
  • Transferring suppression from a supervised representative to a visited context requires an intent-sufficient condition that bounds changes in unsafe mass.The condition applies only at covered contexts, and the theorem also assumes every unsafe turn includes an emitted unsafe token at a safety-critical position.Source: §3 · p. 3
  • When leakage is zero and the base-policy risk budget is positive, the bound contracts that budget if and only if covered share is positive and suppression exceeds twice the transfer slack.The bound separates covered mass, suppression, transfer slack, and leakage; the budget evaluates the base policy on contexts visited by the new policy.Source: §3 · pp. 3–4
  • The contraction result does not establish reduced risk relative to the base policy on its own trajectories.Coverage, transfer slack, and leakage can depend on the attack horizon.Source: §3 · p. 4

4 METHODOLOGY

TRACE combines preferred-response token weighting and rejected-response proxy erasure, with a gradient-norm penalty using the same preference batch. Its theoretical analysis connects local suppression at selected contexts to multi-turn trajectory risk under stated coverage and suppression conditions.

4 METHODOLOGY

  • TRACE combines preferred-branch TAW, rejected-branch RLPE, and a gradient-norm penalty computed on the same preference batch.The penalty avoids an additional retain dataset.Source: §4 · p. 4, §4.1 · p. 4, §4.2 · p. 6

4.1 TRACE ALIGNMENT OBJECTIVE

  • TAW weights preferred tokens using discounted reference–ablated contrasts from the current token onward, allowing later refusal evidence to affect earlier-token weights.The discount sets the effective horizon, from immediate contrasts as λ approaches 0 to an undiscounted return as λ approaches 1; returns can reverse a myopic token ranking.Source: §4.1 · pp. 4–5
  • TAW’s token weights remain positive and have an explicit within-response ratio bound under the stated parameter constraints.The weights are clipped and normalized within each response after preliminary response-level and policy-relative factors are applied.Source: §4.1 · p. 5
  • RLPE selects up to k high-gap rejected-response positions and includes policy-ranked proxy tokens in the target, using the gap as a contrastive signal rather than a harmfulness label.The selected-position budget also sets the number of policy-selected alternatives; targets are recomputed without gradient.Source: §4.1 · pp. 5–6

4.3 THEORETICAL ANALYSIS OF MULTI-TURN ROBUSTNESS

  • The RLPE aggregate loss does not guarantee that every proxy probability or total unsafe mass decreases; the stated risk bound requires additional tokenwise conditions.The bound connects proxy suppression to trajectory risk under Theorem 3.3’s assumptions, including suppression of unsafe proxy tokens and bounds on remaining unsafe tokens.Source: §4.1 · p. 6, §4.3 · p. 7

4.2 RETAIN-SET-FREE MODEL RETENTION VIA GRADIENT-NORM PENALIZATION

  • The gradient-norm penalty is motivated by a bound relating a plain alignment-gradient step to changes in utility-relevant outputs; its effects are evaluated empirically under AdamW.The first-order approximation uses an extra gradient near the alignment-gradient direction; at fixed mixing weight, its error scales as O(αξ^2).Source: §4.2 · p. 6

4.3 THEORETICAL ANALYSIS OF MULTI-TURN ROBUSTNESS

  • Proxy coverage measures captured base-policy unsafe mass, while the gate determines which contexts contribute to the bound; enlarging a proxy set can raise its coverage without guaranteeing a higher minimum across selected contexts.At full proxy coverage, the stated tokenwise condition reduces to Δtok > 2ε; the resulting trajectory-level attack success probability is bounded by min{1, RT}.Source: §4.3 · p. 7

5 EXPERIMENTS

Across five models and seven multi-turn protocols, TRACE has lower attack success than the strongest per-model baselines and leads in every evaluated model–protocol pair. The experiments use single-turn preference tuples, and ablations compare its alignment components and retention penalty.

5.1 SETTINGS OF EXPERIMENTS AND EVALUATIONS

  • The 400 training tuples are single-turn; seven evaluation protocols include two fixed-turn replay attacks and five adaptive attacks.The evaluations cover five open-weight models; attack success is judged if any turn succeeds.Source: §5.1 · p. 7

5.2 EXPERIMENTAL RESULTS AND ANALYSIS

  • On Gemma-3-1B, Llama-3.1-8B, and Qwen-3.5-9B, TRACE’s mean ASR is 26.55%, 27.14%, and 27.31% versus 53.13%, 58.91%, and 49.90% for each model’s strongest baseline.Table 1 covers seven multi-turn attacks and utility measures for these three models.Source: Table 1 · p. 8, §5.2 · p. 8
  • 28.98% mean multi-turn ASR for TRACE versus 51.88% for the lowest-mean baseline selected separately per model; TRACE leads in all 35 model–protocol pairs.Both summaries weight models and protocols equally.Source: §5.2 · p. 8
  • On Qwen-3.5-27B, TRACE’s mean ASR is 29.49% versus 42.76% for the evaluated W-DOOR variant; on Qwen-3.6-27B, TRACE reaches 34.43% while baselines reach 54.68–59.40%.Table 2 reports the Qwen-3.5-27B LoRA results.Source: §5.2 · p. 8, Table 2 · p. 9
  • At ξ = 0.01, safe-prompt refusal is 12.15% versus W-DOOR’s 11.75%, while mean multi-turn ASR is 31.32% versus 59.28%.The passage reports this as part of a refusal sweep.Source: §5.2 · p. 9

5.3 ABLATION STUDY

  • Removing TAW raises mean ASR from 27.14% to 50.37%, while removing RLPE raises it to 34.91%; omitting GNP improves ASR but worsens both utility scores.Replacing GNP with a retain loss on 400 Alpaca examples yields 36.50% mean ASR and MMLU/HellaSwag scores of 66.08%/77.15%.Source: §5.3 · p. 9, Table 3 · p. 9

6 CONCLUSION

TRACE combines continuation-weighted learning on preferred responses with localized proxy erasure on rejected responses, using single-turn preference data. Its analysis connects local suppression to trajectory risk under coverage, transfer, and leakage conditions, while experiments report the lowest ASR across model–attack pairs with less utility degradation than the original checkpoints.

6 CONCLUSION

  • TAW weights preferred-response tokens using discounted reference–ablated contrasts, while RLPE targets selected rejected-response positions with policy-ranked alternatives and GNP regularizes without a separate retain dataset.Later refusal-related evidence can contribute to earlier token weights.Source: §6 · pp. 9–10
  • TRACE achieves the lowest reported ASR across all model–attack pairs, with less utility degradation than the original checkpoints.Same-scorer and component ablations support the objective design, while a radius sweep exposes a robustness–utility–refusal trade-off.Source: §6 · p. 10

C.1.1 PROOFS FOR PROPOSITION 3.1

  • A favorable response-level margin can leave an unsafe token transition unchanged, and static preference loss cannot bound trajectory risk on histories outside its training support.The proof constructs policies with identical preference loss on training prompts but trajectory risks of T and 0.Source: §C.1.1 · pp. 18–19

C.1.4 READING EQ. (4)

  • The trajectory-risk bound relates local suppression at supervised contexts to a base-policy budget through coverage, transfer slack, and leakage.The comparison budget evaluates the base policy on contexts visited by the trained policy, not on the base policy’s own trajectories.Source: §C.1.4 · p. 20

C.1.5 WHY ASSUMPTION 3.2 IS STATED ON THE LOG SCALE

  • On the log scale, transfer yields a multiplicative comparison and the contraction condition becomes ∆ > 2ε, whereas additive slack must be smaller than covered unsafe mass.The covered unsafe mass is the quantity a multi-turn attack inflates.Source: §C.1.5 · p. 20

C.2.2 BOUNDED RELATIVE TOKEN WEIGHT

  • The token-weight ratio is bounded within each response; at ψ = 1, Amax = 3, and ν = 0.5, the bound is 4.43 for χ = 5 and about 538 for χ = 1.The bound applies to response-normalized weights as well.Source: §C.2.2 · p. 21

C.4.1 PROOF FOR THEOREM 4.4

The proof converts tokenwise suppression on gated positions into a context-level factor, then applies the trajectory-risk bound. Its contraction condition depends on proxy coverage and transfer slack.

C.4.1 PROOF FOR THEOREM 4.4

  • Strict budget contraction requires positive coverage and a tokenwise suppression condition strong enough to overcome transfer slack.The proof evaluates the contraction using the bound from Eq. (19).Source: §C.4.1 · p. 23
  • The tokenwise condition in Eq. (20) is κ(1−e−∆tok) > 1−e−2ε; at κ = 1, this condition reads ∆tok > 2ε.Source: §C.4.1 · p. 23
  • The transfer factor e^2ε cannot be folded inside the bracket because that expression is smaller and does not follow from the proof.The expressions coincide when κ = 1 or ε = 0.Source: §C.4.1 · p. 23

C.4.2 PROOF FOR COROLLARY 4.5

This material interprets TRACE’s token weights as discounted returns, shows how they can reverse a myopic token ranking, and describes their standardization. It also cautions that contraction of a risk budget alone need not certify lower conversation-level ASR.

C.4.2 PROOF FOR COROLLARY 4.5

  • Contraction relative to the token-mass budget does not by itself establish improvement over the trivial trajectory-risk or ASR bounds.An informative ASR certificate requires the bound’s right-hand side to be below 1.Source: §C.4.2 · p. 23

D.1 REWARD- AND ADVANTAGE-WEIGHTED REGRESSION IN THE TOKEN-LEVEL MDP

  • In the token-level MDP, each step advantage is a reward and the demonstrated continuation’s return is its current reward plus discounted future rewards.TAW uses this demonstrated return rather than an expectation over alternative continuations.Source: §D.1 · p. 23

D.2 CONSTRUCTIVE SEPARATION BETWEEN MYOPIC AND RETURN-BASED WEIGHTING

  • Discounted returns can rank an earlier token above a later token with a greater immediate advantage, unlike a myopic non-decreasing weight.In the construction, the onset ranks first when its advantage exceeds (1−λ) times the next token’s advantage.Source: §D.2 · p. 24

D.3 THE POSITIONAL PRIOR OF A DISCOUNTED RETURN

  • 40.26% versus 27.14% mean multi-turn ASR on Llama-3.1-8B at λ = 0.9 versus λ = 0.5, respectively.When the two scorers coincide, r ≡ 0, so G ≡ 0 and wtraj ≡ 1.Source: §D.3 · p. 24

D.4 DESIGN OF THE STANDARDIZATION

  • Within each preferred response, TRACE standardizes returns with the scaled median absolute deviation because short responses and correlated returns can make mean-and-standard-deviation scaling sensitive to a few tokens.The scaled median absolute deviation is computed separately within each response.Source: §D.4 · p. 24
  • At χ = 5, token weights range from 0.463 to 1.537; at χ = 1, they range from 0.005 to 1.995, shifting weight toward continuations with more refusal-attributable advantage.Standardized returns are clipped before the squash.Source: §D.4 · p. 24

D.5 TRACE TRAINING ALGORITHM

Algorithm 1 constructs preferred-token weights and rejected-response proxy targets on each minibatch, then updates the policy using a fixed alignment surrogate. The gradient-norm penalty uses the same batch, and frozen scorers remain unchanged.

D.5 TRACE TRAINING ALGORITHM

  • Each update evaluates the same fixed alignment surrogate at two parameter points using the same minibatch.Weights, gates, and targets are detached and held fixed during these evaluations.Source: §D.5 · p. 26
  • For preferred responses, the algorithm computes frozen-scorer advantages, discounted returns, and standardized token weights before the alignment update.The returns are computed backward from the end of each response.Source: §D.5 · p. 26
  • For rejected responses, the algorithm constructs gates and proxy targets without gradients; only trainable parameters are perturbed, leaving frozen scorers unchanged.The kproxy = 0 ablation uses a separate NPO branch, while ξ = 0 disables GNP.Source: §D.5 · p. 26

E WHAT THE BOUNDS MEASURE

The bounds describe when local suppression at supervised single-turn contexts can constrain risk on visited multi-turn histories, while experiments measure conversation-level ASR rather than the bound’s conditions. Training uses 400 single-turn preference tuples, not multi-turn dialogues.

E WHAT THE BOUNDS MEASURE

  • Theorems 3.3 and 4.4 bound risk on visited attack contexts under sufficient local unsafe-token suppression, but experiments do not estimate coverage, transfer slack, leakage, or proxy suppression.The bound’s tokenwise suppression condition is an additional premise; the shared k sweep combines changes to position selection, proxy support, and loss normalization.Source: E WHAT THE BOUNDS MEASURE · p. 26
  • With gates and targets fixed, the local gradient analysis explains the tokenwise suppression premise, while training gradients also depend on shared parameters, TAW, and the gradient-norm penalty.A direct step on a selected logit block lowers its proxy logit when m_j < λ_proxy and raises it when m_j > λ_proxy.Source: E WHAT THE BOUNDS MEASURE · p. 27

F.1 TRAINING DATASET

  • 400 single-turn preference tuples pair harmful requests with preferred safe and rejected responses; no multi-turn dialogue is constructed or used for training.The 400 preferred targets contain 222 distinct refusals, and 21 rejected targets are empty.Source: §F.1 · p. 27

F.2 TRAINING SETTINGS

Training uses full fine-tuning for models from 1B to 9B and LoRA for the two 27B Qwen models, which do not fit on one H200 under full fine-tuning. TRACE uses frozen scoring checkpoints and no retain data.

F.2 TRAINING SETTINGS

  • Models from 1B to 9B are fully fine-tuned; Qwen-3.5-27B and Qwen-3.6-27B use rank-16 LoRA adapters over 4-bit bases because they do not fit on one H200 under full fine-tuning.For evaluation, adapters are merged into the base weights while other tensors are copied unchanged.Source: §F.2 · p. 27
  • TRACE scores tokens with frozen reference and refusal-ablated checkpoints; replacing the ablated model with the reference would make its token returns zero and weights uniform.The ablated checkpoint is not trained; both scoring models are queried only for token log-probabilities.Source: §F.2 · p. 27
  • TRACE uses no retain data, while DOOR and W-DOOR use a retain-set SFT term under full fine-tuning; baseline gradient clipping is 1.0 and TRACE clipping is disabled.Under LoRA, the evaluated W-DOOR configuration disables the retain term and uses constant token weights.Source: §F.2 · p. 28
  • The default shared position and proxy budget is kproxy = 10, and the reported results average five experimental runs using training seed 42.Hyperparameter sensitivity studies use Llama-3.1-8B; additional models assess transfer across backbones.Source: §F.2 · pp. 28–29

F.3 EVALUATION DATA AND ATTACK METHODS

Evaluation covers seven multi-turn attack protocols, with success defined at the conversation level when any turn succeeds. Baselines share preference data and training epochs, though the evaluated LoRA W-DOOR configuration uses uniform token weights without a retain loss.

F.3 EVALUATION DATA AND ATTACK METHODS

  • Protocols include fixed-turn replay, dataset-goal generation, and live adaptive attacks; ActorAttack conversations are generated separately for each checkpoint.Attacker-side generation uses WizardLM-2-8x22B locally, while RACE reports one aggregate result from a 450-item pool.Source: §F.3 · p. 29, Table 8 · p. 29
  • GPT-4o-mini judges full conversations, while MHJ uses a turn-level HarmBench classifier; on MHJ, TRACE improves over the best evaluated baseline by 21.93–39.35 percentage points across five models.Automated judgments can produce false positives, and the examples do not establish judge error rates.Source: §F.3 · p. 29
  • A test attack succeeds if any turn is judged successful, and the reported mean multi-turn ASR averages the seven protocol percentages equally across five runs.A later refusal does not reverse an earlier successful turn.Source: §F.3 · p. 29

F.4 BASELINES

  • Baselines use the same preference data and training epochs, but the evaluated LoRA W-DOOR configuration has uniform token weights and no retain loss.This comparison is against that configuration, not the full weighted, retain-set version.Source: §F.4 · p. 30

G.1 EXPERIMENTS RESULTS ON QWEN-3.6

On Qwen-3.6-27B, TRACE lowers mean multi-turn ASR and leads all seven attacks, with small utility changes. Llama-3.1-8B sweeps show non-monotonic robustness across the discount, proxy budget, and gradient-norm penalty settings.

G.1 EXPERIMENTS RESULTS ON QWEN-3.6

  • 34.43% mean multi-turn ASR for TRACE on Qwen-3.6-27B, versus 47.14% originally and 54.68–59.40% for baselines; TRACE leads all seven attacks.Against the best baseline, ASR falls from 55.00% to 28.06% on ActorAttack and from 76.10% to 50.94% on X-Teaming; MMLU and HellaSwag move by less than 0.2 points.Source: §G.1 · p. 30

G.2 EXPLORATION OF λ SETTINGS OF LTAW

  • 27.14% mean multi-turn ASR at λ = 0.5, down from 43.55% at λ = 0 and rising to 40.26% at λ = 0.9; MMLU and HellaSwag also decline across the sweep.From λ = 0.1 to λ = 0.9, MMLU drops from 67.06% to 66.31% and HellaSwag from 78.68% to 77.76%.Source: §G.2 · pp. 30–31, Table 10 · p. 31

G.3 EXPLORATION OF kproxy SETTINGS OF LRLPE

  • 27.14% mean multi-turn ASR at k = 10, versus 38.65% for the k = 0 NPO special case and 46.60% at k = 20.For k > 0, the sweep jointly changes position selection, proxy support, and sometimes loss normalization; k = 0 is not an isolated proxy-smoothing ablation.Source: §G.3 · p. 31, Table 11 · p. 31

G.4 EXPLORATION OF ξ SETTINGS OF GRADIENT NORM

  • 20.89% mean multi-turn ASR at ξ = 0, but MMLU and HellaSwag fall to 65.42% and 75.36%; ξ = 0.01 keeps both within 0.2 points of the original model at 31.32% ASR.From ξ = 0.01 to ξ = 0.2, mean ASR falls to 23.63% while MMLU declines from 67.63% to 66.16%.Source: §G.4 · p. 31

G.5 ANALYSIS OF OVER-REFUSAL

The penalty-radius sweep on Llama-3.1-8B exposes a trade-off between multi-turn robustness and refusals of safe prompts. The accompanying examples include both over-refusal and partial harmful assistance followed by a refusal.

G.5 ANALYSIS OF OVER-REFUSAL

  • 47.52% safe-prompt refusal at ξ = 0.2 accompanies mean multi-turn ASR falling from 31.32% at ξ = 0.01 to 23.63% at ξ = 0.2.The measurements are for Llama-3.1-8B; the refusal rate is measured on safe XSTest prompts.Source: §G.5 · p. 31, Table 13 · p. 32
  • At ξ = 0.01, TRACE refuses about as often as W-DOOR (12.15% against 11.75%), while mean multi-turn ASR is 31.32% against 59.28%.Source: §G.5 · p. 32

G.6 EXAMPLE CONVERSATIONS AND LLM JUDGE OUTPUTS

  • A political-criticism example shows the target refusing a request the paper characterizes as not unsafe, while the judge labels the attack unsuccessful.The paper cautions that the judge label does not establish that the requested discussion was unsafe.Source: §G.6 · pp. 32–33

Defense Successful Case: Multi-Turn Jailbreak Example with GPT Judge Evaluation

  • One judge-labeled success example actually refuses help with a threat, while a separate multi-turn sexual-content sequence ends in refusal.The target refuses to draft a threatening letter and later refuses to provide seductive phrases.Source: Defense Successful Case: Multi-Turn… · pp. 34–36

Defense Failure Case: Multi-Turn Jailbreak Example with GPT Judge Evaluation

The failure example shows harmful suggestions appearing before a refusal. A training example contrasts a brief preferred refusal with a detailed harmful rejected response.

Defense Failure Case: Multi-Turn Jailbreak Example with GPT Judge Evaluation

  • The response first lists sarcasm and passive-aggressive tactics, then refuses further harmful guidance, and the judge labels the attack successful.The judge explanation treats the initial suggestions as potentially encouraging harmful interpersonal behavior despite the later disclaimer.Source: Defense Failure Case: Multi-Turn… · p. 39

Training Data Example(Preference Pair: Harmful Request with Chosen / Rejected Responses)

  • A bomb-making preference pair contrasts a brief chosen refusal with a multi-step harmful rejected response.The chosen response says it cannot fulfill the request; the rejected response continues with procedural steps.Source: Training Data Example(Preference Pair:… · pp. 41–42
Loading 2610.01323v1…