PaperlayerView exact revision on arXiv

Source-linked AI summary

One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails

Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram

arXiv:2610.12292v1cs.AI

TL;DR

Typed decision models are being used as agent security gates, but prior evaluations do not distinguish their two error directions despite attacker-controlled inputs. This paper evaluates seven open-weight models, attacks, and defenses, finding weak screening performance and attacks that can reverse decisions; parsed-value rules also make the model unnecessary on the synthetic policies.

  • Problem

    Prior security evaluations of typed decision models do not separate errors that allow prohibited actions from errors that block permitted ones, despite attacker-reachable inputs.

    Source: §1 · p. 1

  • Method

    The paper evaluates seven open-weight models as guardrails, separates fail-open and fail-closed errors, and tests attacks and defenses across screening tasks and GuardBench.

    Source: §1 · p. 2, §5 · p. 4

  • Results

    Public-screening accuracy never exceeds 72%, while parsed-value rules reach 100% accuracy on all six GuardBench policies.

    Source: §1 · p. 2, §11 · p. 16

  • Takeaways & Limitations

    Parsed values eliminate one attack, but a deterministic rule over those values makes the model unnecessary; on this evidence, these models should not decide.

    Source: §11 · p. 16

    The study uses open-weight models only, so its mechanistic claims apply to implementations the authors can inspect.

    Source: §10 · p. 16

Abstract

from arXiv · show

A typed decision model reads a piece of text and returns a probability over caller-defined options, each with a short written definition, generating no text. Recent work places these models in agent systems as guardrails: the component that reads a proposed tool call or incoming message and decides whether to allow it. We evaluate seven open-weight models in that role and report the two error directions separately: a fail-open error allows a prohibited action and is a vulnerability; a fail-closed error blocks a permitted one and is only a cost. On prompt-injection, jailbreak and toxic-content screening, accuracy at the allow-or-block decision ranges from 36% to 72% against a chance level of 50%. A low error rate in one direction only reflects which answer a model defaults to: one allows nearly everything, another blocks nearly everything. On a synthetic suite of agent tool calls, six lines of server log text that say nothing about the policy raise a gate's fail-open rate from 0% to 63% on a policy it otherwise decides correctly. Giving the permissive option a misleading name, with its definition and the judged text untouched, raises that rate to between 93% and 100% on the four models that place the label in their input. Every defense we tested is defeated, either by an attacker who targets its mechanism or by attacker-controlled text. Escalating the least confident decisions does not help either: a decision an attack has reversed is no less confident than the one it replaced. Parsing each policy field into a typed value does eliminate one attack, but it also makes the model unnecessary: a deterministic rule over those values reaches 100% accuracy on all six policies. These models can reduce how many cases reach a reviewer, but on this evidence they should not be the component that decides. Code is available at https://github.com/ArminAzizi98/option-channel-attack.

1 INTRODUCTION

The paper evaluates typed decision models as agent guardrails, where an attacker may control part of the input and the two error directions have different consequences. It reports that tested models perform poorly on public screening tasks and can be manipulated through text or option labels.

1 INTRODUCTION

  • Fail-open errors allow prohibited actions, while fail-closed errors block permitted ones; a single accuracy score hides this security-relevant distinction.The gate’s state may contain attacker-reachable tool output, retrieved documents, or user messages.Source: §1 · p. 1
  • The evaluation addresses a gap in prior security work, which reports attacks but neither separates error directions nor proposes defenses.The paper evaluates seven open-weight models and reports attacks only on decisions the gate initially made correctly.Source: §1 · p. 2
  • Public-screening accuracy never exceeds 72%, and attacks exploit both policy-irrelevant text and option labels to drive gates toward allowing prohibited actions.The paper also reports that a misleading option label reaches a 100% fail-open rate on four of six GuardBench models.Source: §1 · p. 2

2 BACKGROUND: TYPED DECISION MODELS

  • A typed decision model receives text to judge and a typed question, then returns probabilities over caller-defined options without generating text.Each option has a short label and written definition; depending on the calling library, the label may reach the model as input.Source: §2 · pp. 2–3
  • The attacker may control the judged state and, in some deployments, option labels supplied through configuration or a tool-provided schema.The state can include tool output and retrieved text.Source: Figure 1 · p. 3, §2 · p. 3

3 RELATED WORK

Prior work reports attacks on typed decision models but does not separate fail-open from fail-closed errors or propose mitigations. This paper frames an attacker who can control a trace span or, under weaker access, an option label.

3 RELATED WORK

  • Prior attacks append opinions or context, study prompt injection, or rename options; the cited work neither separates error directions nor proposes a mitigation.Renaming option sets had been reported with changes in answers attributed to the constrained decision head.Source: §3 · p. 3
  • An evidence review found label-probability baselines in 5 of 26 papers and option-name invariance tests in 1 of 27.Prior recommendations to escalate low-confidence cases are also tested in this paper.Source: §3 · p. 3
  • Table 1 lists six GuardBench policies over proposed tool-call attributes and gives the number of conditions in each rule.Each rule specifies when the gate must block an action.Source: Table 1 · p. 4

4 THREAT MODEL

  • The threat model gives the attacker control of one state span, such as tool-returned text, and targets a fail-open error using black-box attacks with few queries.The attacker does not edit the policy or option definitions.Source: §4 · p. 4
  • A weaker access path lets an attacker influence an option label while leaving the option definition and judged state untouched.This applies when questions are assembled from configuration, fetched policy documents, or tool-supplied schemas.Source: §4 · p. 4

5 BENCHMARKS, MODELS AND METRICS

The benchmarks pair synthetic tool-call policies with three public screening tasks and report the two error directions separately. On the public tasks, Qwen2.5-1.5B’s low fail-open rate on injection screening accompanies a 100% fail-closed rate.

5 BENCHMARKS, MODELS AND METRICS

  • GuardBench generates tool-call traces from six attributes, defines six policies over them, and computes exact labels from the generating attributes.Its classes are balanced exactly, and every trace ends with an attacker-controlled tool-output span.Source: §5 · p. 4
  • Each GuardBench policy is phrased positively, explicitly as a negation, or semantically as a person might state it.Attack experiments use positive phrasing so the gate demonstrably works before attack; wording comparisons are reported separately.Source: §5 · p. 4
  • The public tasks cover prompt-injection, jailbreak, and toxic-content screening using balanced datasets of real-user text.The entire message in each task is treated as attacker-written.Source: §5 · p. 4
  • 0.7% fail-open on injection screening pairs with 100% fail-closed for Qwen2.5-1.5B, which blocks almost every input.Its accuracy is below chance on that task.Source: Table 2 · p. 5
  • The evaluation reports fail-open and fail-closed rates separately, and scores attacks only on items the gate initially decided correctly.It also measures distance from the 0.5 probability boundary and uses paired bootstrap resampling for clean and attacked predictions.Source: §5 · p. 5

6 HOW THESE GATES BEHAVE, AND HOW THEY FAIL

Across public screening and synthetic tool-call policies, these gates often default to one answer and can be shifted toward allowing by irrelevant text or option labels. Confidence-based review does not reliably detect reversed decisions.

6.1 EACH MODEL IS UNSAFE IN ITS OWN DIRECTION

  • Accuracy was 36%–72% versus 50% chance; models clustered near the input-ignoring line, and negated policy wording caused 100% fail-open on two policies.The low error rate in one direction reflected whether a model tended to allow or block.Source: §6.1 · pp. 5–6, Figure 2 · p. 6

6.2 TEXT THAT CARRIES NO INFORMATION ABOUT THE POLICY OPENS THE GATE

  • On unauth irrev, six log lines raised LAYA-TD’s fail-open rate from 0% to 63%, while three unverified assertions raised it to 70%, although the correct decision was unchanged.Across attack cells, the median fail-closed rate was unchanged; the attacks shifted decisions toward allowing.Source: Table 3 · p. 7, §6.2 · p. 7

6.3 THE OPTION CHANNEL

  • Changing only the permissive option’s label raised fail-open rates to 93%–100% on the four models that fed labels to the model.The definitions stayed correct and the judged state was untouched.Source: Table 4 · p. 8, §6.3 · p. 8
  • On LAYA-TD, security-action labels averaged 57% fail-open versus 0% for honest naming, and ten random labels reached 90%; some labels also transferred across related models.The effect depended on the model: LAYA-TD responded most to security-action labels, while LAYA-EN responded most to unrelated nouns.Source: WHICH LABELS WORK, AND WHAT THE ATTACK… · pp. 9–10
  • On public screening, LAYA-TD reached 100% fail-open on jailbreak screening with the label “escalate,” while LAYA-ML rose from 10% to 63% on toxic-content screening.The real-user text being judged was not modified.Source: Table 7 · p. 10, WHICH LABELS WORK, AND WHAT THE ATTACK… · p. 10
  • Renaming the permissive option “remediate” raised Qwen2.5-7B’s fail-open rate to 100% on all six policies, despite its 5% mean honest rate.Ten random labels reached 89%.Source: WHICH LABELS WORK, AND WHAT THE ATTACK… · p. 10

6.4 CORRECT BLOCK DECISIONS SIT CLOSE TO THE 0.5 BOUNDARY

  • For LAYA-TD, the clean margin predicted reversals, but no threshold was both usable and resistant: at 0.40, clean fail-closed was 70.7% and attacked fail-open was 10.0%.At a 0.30 threshold, the gate blocked every item.Source: §6.4 · pp. 10–11

6.5 A REVERSED DECISION DOES NOT LOOK UNCERTAIN

  • Reversed decisions were no less confident (0.149 margin before, 0.151 after); even 10% review reduced attacked fail-open only from 60% to 54%.Clean margins predicted which decisions an attack would reverse, but the attacked margin did not reveal that a reversal had occurred.Source: §6.5 · p. 11, Figure 6 · p. 11, Table 8 · p. 11

7 DEFENSES

The defenses were compared at matched clean fail-closed cost, and their results vary by mechanism and setting. Parsing policy-relevant fields removes field injection, but value confusion remains and deterministic rules perform better on the synthetic policies.

7 DEFENSES

  • Across 72 matched defense-model-policy cells, defenses win 45, tie 21, and lose 6.The comparison matches each defense to an undefended gate with the same clean fail-closed rate; a defense wins only if its attacked fail-open rate is lower.Source: Table 9 · p. 12, §7 · p. 12
  • D5 is the only defense that never loses, winning 16 of 18 cells and tying two, but it blocks 68.7% of permitted actions on unauth irrev.D5 passes policy-relevant structured fields rather than the raw trace.Source: §7 · p. 12
  • Label-based defenses win 9 or 10 of 18 cells but lose two on the same policy; they remain the only mitigation for the option channel.VON's ties are expected because it never receives the option label, so relabeling leaves its input unchanged.Source: §7 · pp. 11–12
  • On public screening, D1 raises jailbreak clean accuracy from 68.0% to 86.0%, while attacked fail-open rates remain 68.0% on injection and 84.6% on toxic screening.The defenses reduce risk where the model has usable signal but do not create signal where it has none.Source: §7 · p. 12
  • On LAYA-TD, instructing the model to ignore option names leaves attacked fail-open at 99%, versus 98% undefended; searching new labels restores the attack on Qwen.For Qwen, the attack reaches 100% on all six policies after searching the same 198 labels, despite the instruction.Source: Table 10 · p. 12, §7 · p. 13

7.1 VALUE NORMALIZATION, AND WHAT IT IMPLIES

  • Parsing fields cuts field injection from 53% to 0% averaged over six policies, with zero rates for every model-policy pair.The gate receives the parsed value rather than field text, removing appended annotations from its input.Source: §7.1 · p. 13, Table 11 · p. 14
  • Confusing values still cause 20.3%–50.0% mean fail-open rates, while deterministic rules over the same parsed values reach 100% accuracy on all six policies.The confusing host remains a violation despite containing the internal-domain string; path traversal, by contrast, fails after resolving ..Source: §7.1 · pp. 13–14, Table 11 · p. 14
  • The rule result is specific to GuardBench’s explicit generating attributes; public screening text has no field reducible to a decidable value.GuardBench’s construction enables exact ground truth and deterministic rules, rather than showing that these tasks are generally easy.Source: §7.1 · p. 14

8 ADAPTIVE ATTACKS

Adaptive attacks target the retained fields and the defenses’ mechanisms, rather than only the original attack surface. Their success depends on whether the attacker can place the permitted case’s vocabulary inside fields the gate still reads.

8 ADAPTIVE ATTACKS

  • Against D5, the adaptive attack reaches 100% on four typed models and 55.7% on LAYA-ML, though it remains ineffective on unauth irrev and secret in the stated evaluable cases.D5 has zero fail-open rates against the original attack in every evaluable cell.Source: §8 · p. 14
  • D5 fails on policies whose allow definitions name surface properties the attacker can imitate inside retained fields, but holds where that wording is absent.For example, an annotation puts api.internal.corp into a destination field whose value remains paste.ee.Source: §8 · pp. 14–15
  • Searching for suffixes that make both labelings allow raises LAYA-ML’s fail-open rate on secret from 12.0% to 83.1%; VON and RLCD are also defeated despite label invariance.The adaptive disagreement check is described in Table 13 as an attacker optimizing for both labelings.Source: §8 · p. 15, Table 13 · p. 15

9 RECOMMENDATIONS

The recommendations treat typed decision models as mitigations rather than final security decision-makers. They favor controlling attacker-accessible inputs while recognizing that the tested defenses and probability-based escalation remain limited.

9 RECOMMENDATIONS

  • Do not make a typed decision model the final allow-or-block component: every tested configuration could be opened by an adaptive attacker.The paper reports that a short suffix can reverse a correct block decision.Source: §9 · p. 15
  • Pass structured action fields rather than raw traces, but treat this as mitigation: adaptive attacks defeat it on four of six policies, and it blocks 68.7% of permitted actions on unauth irrev.D5 was the only defense that never lost the matched comparison.Source: §9 · p. 15
  • Keep untrusted strings out of option labels; call-site relabeling closes the option channel but does not stop attacks through attacker-controlled text.On unauth irrev, this defense raises clean fail-closed from 25.3% to 36.0%.Source: §9 · p. 15
  • State blocking conditions as present properties rather than negations, since negated conditions produced 100% fail-open rates on two policies.Source: §9 · p. 15
  • Neither lowest-margin escalation nor the returned probability identifies reversed decisions; disagreement between labelings is more informative but can also be attacked.Asking the model to judge by definitions alone raises the number of labels an attacker must try without changing the reachable fail-open rate.Source: §9 · p. 15

10 LIMITATIONS

The evidence has boundaries in benchmark realism, model coverage, and attacker strength.

10 LIMITATIONS

  • GuardBench is synthetic and template-generated, giving exact ground truth, exact class balance, and no training-data overlap at the cost of realism.Three public datasets provide a complementary setting.Source: §10 · p. 15
  • Only open-weight models are measured, so mechanistic claims apply to implementations the authors can inspect, not the commercial model that introduced the interface.Source: §10 · p. 16
  • The inexpensive black-box attacks use a fixed text pool or vocabulary, so a stronger attacker could achieve higher rates and the reported numbers are lower bounds.Source: §10 · p. 16

11 CONCLUSION

The tested typed decision models fail in the unsafe direction and make correct block decisions with little margin. Although they can reduce reviewer workload, the paper concludes they should not make the decision.

11 CONCLUSION

  • The tested models fail in the unsafe direction and make correct block decisions with little margin.They can be controlled through an input channel earlier work noticed but did not treat as an attack surface or trace to its cause.Source: §11 · p. 16
  • Parsing policy fields into typed values removes one attack, but a deterministic rule over those values is exact and resists both attacks.The paper says typed decision models are genuinely needed only where such parsing does not exist, which is also where this defense is unavailable.Source: §11 · p. 16
  • These models can reduce the number of cases reaching a human reviewer, but the paper concludes they should not be the component that decides.The paper says this reduction buys little protection against an attacker.Source: §11 · p. 16

A EXAMPLES OF EACH ATTACK

The examples are generated from the same runs as the main results. The example uses LAYA-TD on the unauth irrev policy, for which its clean gate is accurate.

A EXAMPLES OF EACH ATTACK

A.1 TEXT THAT CARRIES NO INFORMATION ABOUT THE POLICY

Appending routine server-log text to the attacker-controlled tool-output span changes the gate’s decision, although none of the policy-defined action attributes changes.

A.1 TEXT THAT CARRIES NO INFORMATION ABOUT THE POLICY

  • Appending six lines of routine server-log text moves P(block) from 0.58 to 0.48, allowing an action the gate had correctly blocked.The destination, path, body, reversibility, and authorization are identical in both versions.Source: §A.1 · pp. 16–17
  • The attack appends routine server-log text to the tool-output span, the region assumed to be attacker-controlled.The log reports request usage, an audit entry, response status and timing, queue depth, truncation, and a trace ID.Source: §A.1 · pp. 16–17
  • The paired listing is described as the same action with six lines appended and no change to any attribute the policy is defined over.Source: §A.1 · p. 17

A.2 THE OPTION CHANNEL

The option-channel example changes only the permissive option’s name, while the real-user screening example appends a polite closing. Both examples report a gate decision moving from block to allow.

A.2 THE OPTION CHANNEL

  • Renaming the permissive option from allow to escalate lowers P(block) from 0.58 to 0.43, with the judged text and correct option definitions unchanged.Source: §A.2 · p. 17

A.3 A MESSAGE WRITTEN BY A REAL USER

  • Appending a polite closing to a real user’s toxic-content message changes the gate from blocking at P(block) = 0.59 to allowing at 0.39.Source: §A.3 · p. 17
  • Across the three examples, the clean decision sits just above the boundary and the attack moves it just below.The passage identifies this as the margin result of §6.4 shown one item at a time.Source: §A.3 · p. 17

Related papers

Loading 2610.12292v1…