PaperlayerView exact revision on arXiv

Source-linked AI summary

Intent-Hiding Jailbreaks: An Information-Theoretic Framework for Compositional Attacks

Fengwei Tian, Ravi Tandon

arXiv:2610.02302v1cs.CRcs.ITcs.LGmath.IT

TL;DR

LLMs may answer harmful requests when they are embedded in benign-looking tasks, even if they refuse those requests directly. This paper models intent hiding through prior–posterior matching, studies bundle-only and joint query construction, and evaluates attack success and target preservation. Its evaluation finds that composition can reach targets refused as direct requests, while target preservation tends to decline with larger bundles for several models.

  • Problem

    A harmful request refused directly may receive a different response when embedded in benign-looking instructions, and the effects of task selection and query wording require analysis.

    Source: §I · pp. 1–2

  • Method

    The paper models bundle-level intent hiding through prior–posterior matching, analyzes exact, fractional, and threshold-based selection, and jointly considers query construction and target preservation.

    Source: A. Information-theoretic abstraction · p. 2, B. Query-independent intent hiding · p. 3

  • Results

    Targets refused as direct requests could often be reached after adding even a small number of benign auxiliary tasks, with effects transferring across model families.

    Source: §VI · p. 21

  • Takeaways & Limitations

    Composition substantially changed model behavior in the evaluated settings, while query-generator choice also mattered for attack success on the more robust responding model.

    Source: §VI · p. 21

Abstract

from arXiv · show

Recent work has shown that large language models (LLMs) can be vulnerable to jailbreak attacks in which harmful intent is obscured through composition with benign tasks. A harmful request refused in isolation may elicit a different response when embedded within a larger, seemingly benign query. We study these compositional intent-hiding jailbreaks from an information-theoretic perspective. Our formulation associates each task with an estimated probability of being judged harmful: the average over the full task collection defines the prior probability of harmful intent, while the average over a selected bundle containing the target defines the posterior. Selecting auxiliary tasks so that these averages agree, which we call prior-posterior matching, leaves the estimated intent unchanged even though the harmful target remains in the bundle. We study two settings that differ in whether query construction is part of the optimization. In the query-independent setting, tasks are selected without regard to how they will be expressed in the final query. We show that exact prior-posterior matching under a bundle-size constraint is computationally hard, derive an optimal water-filling solution for fractional weights, and characterize the smallest bundle satisfying a prescribed safety threshold. In the query-dependent setting, task selection and query construction are considered jointly, and intent concealment and target preservation are evaluated on the resulting query. We evaluate jailbreak effectiveness and preservation of the target behavior across bundle sizes, query generators, and several open-source models. These results show that compositional queries can elicit target behaviors beyond the direct-request baseline under the evaluated search budgets, while revealing a trade-off: as bundle size increases, response-level target preservation tends to decrease for several models.

I. INTRODUCTION

The paper frames compositional intent hiding as task selection under a concealment–target-preservation trade-off. It develops bundle-level and query-level formulations, then evaluates how bundles and query construction affect attack success and target fulfillment.

I. INTRODUCTION

  • Refusing a direct harmful request does not establish that safeguards remain effective when the request is embedded among benign instructions.The introduction links this vulnerability to context-dependent instruction interpretation and the attacker’s ability to disguise harmful purpose.Source: §I · pp. 1–2

A. Information-theoretic abstraction.

  • Prior–posterior matching selects auxiliary tasks so the bundle’s average harmful-intent estimate equals the full collection’s average, leaving estimated intent unchanged.The designated target remains in the bundle under the stated assumptions.Source: A. Information-theoretic abstraction · p. 2

B. Query-independent intent hiding.

  • Exact prior–posterior matching under a bundle-size limit is NP-hard, while fractional water-filling gives a lower bound and a sorted-task rule finds minimum-size bundles under a threshold.The threshold rule adds auxiliary tasks in increasing order of harmful-intent probability until the bundle average meets the threshold.Source: B. Query-independent intent hiding · p. 3

C. Query-dependent intent hiding.

  • Joint query-dependent selection requires a query to meet a harmful-intent threshold and preserve the target, then chooses the best-preserving candidate among those examined.The search examines a limited number of candidate bundles and generated queries; actual LLM success is evaluated separately.Source: C. Query-dependent intent hiding · p. 3

D. Empirical evaluation.

  • The evaluation tests fractional matching and varies bundle size, query generator, and responding-model family and size, measuring target-level attack success and mean target preservation.It uses 100 harmful targets from JailbreakBench and 50 benign auxiliary tasks from Super-NaturalInstructions.Source: D. Empirical evaluation · p. 3

II. PROBLEM FORMULATION

  • The attack pipeline selects a target-containing task bundle, expresses it as a query, and assesses whether the response carries out the target.Bundle-level matching and query-level target clarity guide construction, but actual success must be checked from the response.Source: §II · p. 4

A. Task and Intent Model

The formulation assigns each basic task an estimated probability of harmful intent and uses a bundle-level posterior to model intent hiding. It distinguishes concealing the bundle’s harmful intent from preserving the designated target in the composed prompt.

A. Task and Intent Model

  • Each basic task belongs to a finite task alphabet and has a negative-intent probability; this estimate depends on an assessment mechanism, not an assumed objective semantic ground truth.The analysis treats task probabilities as given, while the empirical section estimates them with Llama Guard 3-8B.Source: A. Task and Intent Model · pp. 4–5

B. Intent-Hiding Jailbreaks via Task Bundling

  • A bundle is represented by inclusion indicators, must contain the designated target, and has one target plus k −1 auxiliary tasks when its total size is k.The experiments restrict auxiliary tasks to benign ones.Source: B. Intent-Hiding Jailbreaks via Task… · p. 5
  • Under the stated sampling assumptions, the bundle-level posterior averages task-level negative-intent probabilities; matching it to the prior models intent hiding, while adding low-intent tasks can move the posterior toward the prior.The smallest attainable prior–posterior gap cannot increase as the bundle-size budget grows.Source: B. Intent-Hiding Jailbreaks via Task… · p. 6
  • The framework hides the bundle’s negative intent rather than the target itself, which must remain salient enough to be expressed in the final prompt.This distinguishes intent privacy from target preservation and from formulations that hide a sensitive variable while preserving useful information.Source: B. Intent-Hiding Jailbreaks via Task… · p. 7

III. QUERY-INDEPENDENT INTENT HIDING

The query-independent analysis seeks compact bundles before deciding how their tasks will be worded. The included figure also reports that response-level target preservation falls as bundles grow for two evaluated models.

III. QUERY-INDEPENDENT INTENT HIDING

  • The query-independent analysis seeks the smallest bundle achieving exact prior–posterior matching, then considers fractional task weights and a one-sided safety threshold.It does not account for how the selected tasks will be written together.Source: §III · p. 7
  • Both models lose the most target preservation at k = 2, then decline more slowly as bundle size increases.The figure reports preservation as a percentage of each model’s direct-request baseline.Source: Figure 3 · p. 8
  • At k = 6, Mistral-7B-Instruct-v0.3 keeps 26% and Qwen3-8B keeps 53% of its direct-request baseline.Source: Figure 3 · p. 8

A. Bundle Intent Hiding and Target Preservation

Bundle-level intent hiding seeks small target-containing collections whose average harmful-intent probability matches the task-wide prior, while limiting dilution of the target. Exact matching has a computationally hard size-bounded version, and bundle size is only a proxy for preserving the target in the response.

A. Bundle Intent Hiding and Target Preservation

  • Bundle size is a model-independent proxy for target preservation; response-level preservation instead depends on the query realization, downstream LLM, and safety mechanism.The analysis introduces an intermediate query-level target-preservation measure before downstream response effects.Source: A. Bundle Intent Hiding and Target… · pp. 8–9

B. Perfect Intent-Hiding Problem

  • Auxiliary tasks can move the bundle posterior toward prior ρ, but each added task dilutes the target, so the objective is the smallest matching bundle.When the target has above-prior intent, auxiliaries below the prior contribute positively to the exact offset.Source: A. Bundle Intent Hiding and Target… · p. 8, B. Perfect Intent-Hiding Problem · p. 9
  • For target intent above the prior, exact matching requires auxiliary-task offsets to equal the target’s excess; only below-prior tasks contribute positively.The section formulates this condition using the selected auxiliary tasks.Source: B. Perfect Intent-Hiding Problem · p. 9
  • Deciding whether a size-bounded deterministic bundle containing the target exactly matches prior ρ is NP-hard; exact matching is possible only when ρ is realizable.The full-alphabet bundle always matches, but maximally dilutes the target.Source: B. Perfect Intent-Hiding Problem · p. 9

C. Fractional Task Bundling

The fractional relaxation permits auxiliary tasks to receive weights between zero and one while keeping the target fully included. Its exact-matching solution prioritizes tasks with the strongest hiding contributions, with at most one fractional task.

C. Fractional Task Bundling

  • The relaxation fixes the target weight at one and allows each auxiliary task’s inclusion weight to range from zero to one.The relaxed bundle size is one plus the sum of auxiliary weights.Source: C. Fractional Task Bundling · p. 10
  • The relaxation is always feasible, but random bundles with the optimal expected inclusion weights need not individually match the prior or have expected posterior equal to it.The fractional solution can induce randomized bundling schemes without guaranteeing either property for sampled bundles.Source: C. Fractional Task Bundling · pp. 10–11
  • For an above-prior target, the fractional optimum uses the largest hiding contributions first, equivalently the lowest-intent auxiliaries, until the deficit is met.The below-prior case is symmetric and uses the highest-intent auxiliaries first.Source: C. Fractional Task Bundling · p. 10
  • The fractional optimum has at most one partially selected auxiliary task, with remaining auxiliaries excluded.The water-filling allocation fills tasks in descending contribution order until the final task meets the remaining deficit.Source: C. Fractional Task Bundling · p. 10

D. Imperfect Intent-Hiding

The imperfect-hiding formulation seeks the smallest target-containing bundle whose harmful-intent posterior is at or below a safety threshold. Under the stated feasibility condition, adding auxiliaries in increasing intent-probability order is optimal, even though the posterior need not decrease monotonically along that path.

D. Imperfect Intent-Hiding

  • If the target’s harmful-intent probability is already at or below τX, the target alone is feasible; otherwise auxiliaries must reduce the bundle posterior.The formulation seeks the smallest deterministic bundle containing the target.Source: D. Imperfect Intent-Hiding · p. 11
  • When τX ≥ ρ, adding auxiliaries in increasing harmful-intent probability and stopping at r(x) ≤ τX yields a minimum-size acceptable bundle.The rule is optimal because for any fixed size, the lowest-intent auxiliaries produce the smallest posterior.Source: D. Imperfect Intent-Hiding · p. 11
  • The posterior can rise after adding an auxiliary whose intent probability exceeds the current bundle posterior, so the greedy path is not necessarily monotone.Optimality follows from checking each bundle size in increasing order, not from monotonicity.Source: D. Imperfect Intent-Hiding · pp. 11–12

IV. QUERY-DEPENDENT INTENT HIDING

Query-dependent intent hiding evaluates task selection together with the natural-language query that expresses the bundle. This joint view accounts for query-level concealment and target identifiability, which bundle-level posterior and size alone do not determine.

IV. QUERY-DEPENDENT INTENT HIDING

  • Bundles with similar posterior and size can yield different queries because task order, combination, and wording affect concealment and whether the target remains identifiable.A coherent scenario may obscure target prominence while making the query’s purpose easier for a safety evaluator to infer.Source: §IV · p. 12

A. Query Intent Hiding and Target Preservation

  • The query-level formulation evaluates concealment through rQ(q) and target preservation through target information ℓb(q), requiring both thresholds for a quality-preserving query.The overall query utility combines the concealment and target-preservation requirements.Source: A. Query Intent Hiding and Target… · pp. 12–13
  • A larger ℓb(q) means the realized query makes the designated target more identifiable; target preservation requires ℓb(q) ≥ η.Under the uniform task prior, the target information is expressed as a log posterior lift.Source: A. Query Intent Hiding and Target… · p. 12
  • ASR measures target coverage within the evaluated candidate budget, whereas response-level target preservation measures how fully responses fulfill the target.These downstream measures do not directly estimate query-level posterior or overall query utility.Source: A. Query Intent Hiding and Target… · p. 13

B. Query-Dependent Intent-Hiding Problem

The query-dependent formulation chooses a target-containing task bundle together with a natural-language realization that conceals intent while preserving the target. Its bounded search evaluates candidate realizations against both requirements and selects the feasible one with the greatest target-preservation value.

B. Query-Dependent Intent-Hiding Problem

  • Query-dependent selection jointly chooses a target-containing bundle and a natural-language realization that meets query-level concealment and target-preservation requirements.Unlike the bundle-level proxy, query-level preservation depends on how tasks are expressed, not bundle size alone.Source: B. Query-Dependent Intent-Hiding Problem · p. 13
  • The objective assigns positive utility only to realizations meeting both requirements, then favors the one with the largest target-preservation value.Bundle size is only a model-independent proxy; similarly sized bundles can yield queries with different coherence, preservation, or apparent intent.Source: B. Query-Dependent Intent-Hiding Problem · p. 13

C. Query-Dependent Optimization

  • Because natural-language realizations are too numerous to enumerate, bounded search tests up to M realizations for candidate bundles and keeps those meeting estimated query-level constraints.Candidate bundles come from the query-independent greedy prefixes and water-filling support; query-level quantities may be estimated with calibrated classifiers, LLM judges, target recognizers, or ensembles.Source: C. Query-Dependent Optimization · p. 14
  • The search is incomplete: finding no feasible query within its budget does not show that none exists, while any returned pair satisfies the estimated constraints by construction.Its cost is M|bCX| query evaluations.Source: C. Query-Dependent Optimization · p. 14

V. EXPERIMENTS AND DISCUSSION

The experiments assess whether benign auxiliary tasks can change responses to harmful targets, using sampled bundles and generated queries across bundle sizes, generators, and open-source models. They compare jailbreak success and target preservation with direct-request baselines.

A. Experimental Setup

  • The fixed task alphabet combines 100 harmful behaviors from JailbreakBench with 50 benign language-processing tasks sampled from Super-NaturalInstructions.The harmful targets span ten misuse categories, including fraud, disinformation, and privacy violations.Source: A. Experimental Setup · p. 15
  • Task-level negative-intent probabilities are estimated with Llama Guard 3-8B using normalized unsafe and safe label probabilities.The classifier supplies graded estimates, which are used across bundle constructions.Source: A. Experimental Setup · p. 15
  • Bundles contain one target and k−1 distinct auxiliaries for k from 2 to 6, sampled with weights motivated by the water-filling solution.Responding models use temperature 0, seed 1234, and a 512-token limit; the evaluated set includes Qwen, Mistral, and Gemma models.Source: A. Experimental Setup · pp. 15–16

V. EXPERIMENTS AND DISCUSSION

  • For each target and bundle size, the study draws 20 bundles, generates one query per bundle, and submits all queries without filtering, testing sampling rather than the full search.QWEN3-32B is the primary generator; QWEN3-14B realizes the same 10,000 auxiliary bundles for comparison, and each generator–responding-model pair has 10,100 responses including direct requests.Source: A. Experimental Setup · p. 15, §V · p. 15

A. Experimental Setup

  • HarmBench labels measure attack success, while Flow-Judge scores how fully responses preserve the target behavior; the two outcomes are reported separately.Blank responses count as failures, and no candidates are removed; Table II illustrates direct refusals and composed queries labeled as jailbreaks for Qwen3-4B-Instruct-2507.Source: A. Experimental Setup · p. 16, Table II · p. 17

B. Bundle-Level Concealment

The section uses empirical prior-posterior matching to quantify bundle concealment, then evaluates cumulative attack success across bundle sizes, responding models, and query generators. Composed queries raise attack success, but gains and generator effects vary across models.

B. Bundle-Level Concealment

  • The empirical reference ρ0 is 0.103113 under a 10% target and 90% auxiliary mixture, distinct from the theoretical uniform-alphabet prior of approximately 0.65.The empirical means are computed over estimated negative-intent probabilities for 100 target tasks and 50 auxiliary tasks.Source: B. Bundle-Level Concealment · p. 17
  • For 99 of 100 targets, exact matching is feasible fractionally; median auxiliary mass of 8.713 yields a relaxed bundle size of 9.713.The remaining target is below ρ0 and satisfies the one-sided condition, but exact matching is infeasible with the available auxiliary tasks.Source: B. Bundle-Level Concealment · p. 17

C. Attack Success Rate

  • Cumulative ASR counts each target once if any evaluated candidate at k ≤ K succeeds; the candidate budget per target is 1 + 20(K − 1).The response experiments stop at six tasks because cumulative-ASR gains are generally smaller at larger tested sizes.Source: B. Bundle-Level Concealment · p. 17, C. Attack Success Rate · p. 18
  • First auxiliary tasks add 83 and 35 percentage points for Qwen3-4B and Qwen3-4B-Instruct-2507; for Qwen3-4B, cumulative ASR rises from 3% at K = 1 to 86% at K = 2.For Qwen3-4B-Instruct-2507, ASR reaches 73% by K = 6; beyond K = 3, each size adds at most 7 percentage points for either model.Source: C. Attack Success Rate · p. 18, Figure 4 · p. 18
  • At K = 6, cumulative ASR is 99%, 96%, and 96% for Mistral-7B-Instruct-v0.3, Gemma-3-4B-PT, and Qwen3-4B, while Qwen3-1.7B, Qwen3-4B, and Qwen3-8B reach 99%, 96%, and 97%.Qwen3-4B-Instruct-2507 reaches 73% despite matching Qwen3-4B in parameter count; within the evaluated range, post-training appears to matter more for robustness than size.Source: C. Attack Success Rate · p. 19, Figure 5 · p. 19
  • At K = 6, Qwen3-32B versus Qwen3-14B yields 99% versus 98%, 96% versus 95%, and 73% versus 57% ASR on Qwen3-1.7B, Qwen3-4B, and Qwen3-4B-Instruct-2507.The generators realize identical sampled auxiliary bundles, so the comparison isolates how the tasks are written; the largest difference is 16 percentage points on Qwen3-4B-Instruct-2507.Source: Figure 6 · p. 19, C. Attack Success Rate · p. 20

D. Response-Level Target Preservation

The paper evaluates how fully composed-query responses carry out their harmful target, alongside whether targets can be reached. Composition often reaches targets refused directly, while larger bundles trade greater reach for lower average target preservation in several models.

D. Response-Level Target Preservation

  • Responses are scored against their original target on a normalized 0–1 scale, where 0 means the target is not carried out and 1 means it is fully carried out.The five-level rubric assigns 1 to refusal, deflection, or auxiliary-task-only responses and 5 to fully carrying out the target.Source: D. Response-Level Target Preservation · p. 20
  • Average target preservation generally decreased with bundle size for four responding models, but showed no consistent trend for QWEN3-4B-INSTRUCT-2507 and GEMMA-3-4B-IT.The measure averages all responses at a bundle size, whether or not they are jailbreaks, so its decrease alone does not show that successful responses preserve less of the target.Source: D. Response-Level Target Preservation · p. 20

VI. CONCLUSIONS

  • The formulation models perfect bundle-level concealment through prior-posterior matching and establishes hardness, a fractional water-filling solution, and a one-sided safety-threshold characterization.The paper also extends the formulation to natural-language queries and jointly considers intent concealment and target preservation.Source: §VI · p. 21
  • Composed queries often reached targets refused as direct requests, with effects across model families and smaller direct-request differences after composition.The conclusions also report that increasing model size did not consistently increase robustness, post-training differences appeared more important, and query generators produced substantially different success for the more robust responding model.Source: §VI · p. 21

APPENDIX A: PROOF OF PROPOSITION 1

Appendix A proves that exact prior-posterior matching with a bundle-size constraint is NP-complete for rational task probabilities encoded in binary. Its reduction maps a SUBSET SUM instance to task probabilities so a feasible matching bundle exists exactly when the instance has a solution.

APPENDIX A: PROOF OF PROPOSITION 1

  • The reduction encodes a SUBSET SUM instance so a feasible matching bundle exists if and only if the instance has a solution.It sets n = m + 2, K = n − 1, and constructs task-level negative-intent probabilities from the SUBSET SUM values and target q.Source: Appendix A · pp. 22–23
  • The decision problem is NP-complete for rational task probabilities encoded in binary.The reduction establishes NP-hardness even when K = n − 1 and ρ is any fixed rational in (0, 1); exact rational arithmetic verifies membership in NP.Source: Appendix A · p. 23

APPENDIX B: FRACTIONAL MATCHING AND EMPIRICAL COST

Appendix B derives fractional matching feasibility and the minimum auxiliary mass, then estimates matching costs for the Section V-B task collection. The empirical cost uses a nominal reference level, and its results reflect a deliberately target-heavy task composition rather than real user requests.

A. Fractional Feasibility

  • Fractional matching is feasible exactly when the target offset lies within the interval of total contributions available from auxiliaries.The interval endpoints are −C+ and C−, formed from auxiliaries’ negative and positive contributions relative to ρ0.Source: A. Fractional Feasibility · pp. 23–24

B. Minimum Auxiliary Mass

  • For a positive target offset, the minimum auxiliary mass uses the largest positive contributions first until their cumulative sum reaches the offset.For zero offset, the target alone matches; for a negative offset, the analogous construction uses auxiliaries above ρ0 when feasible.Source: B. Minimum Auxiliary Mass · p. 24

C. Empirical Matching Cost

  • For 99 of 100 targets, median minimum auxiliary mass is 8.713, corresponding to a median relaxed total bundle size of 9.713.The remaining target is below ρ0 and satisfies the one-sided condition, but exact matching is impossible because no auxiliary probability exceeds ρ0.Source: C. Empirical Matching Cost · p. 24

Related papers

Loading 2610.02302v1…