PaperlayerView exact revision on arXiv

Source-linked AI summary

DGF-Bench: A Benchmark for Simulating and Auditing Deception Against Multi-Agent Governance Boards

Jeremy Canale

arXiv:2609.34913v1cs.AIcs.CR

TL;DR

Tool-using agents review evidence that can include content from suppliers or project members with a stake in the decision, exposing governance reviews to indirect prompt injection. DGF-Bench evaluates this risk with paired clean and attacked dossiers; task-aligned imitations of organizational process succeeded where direct orders, false data, and false authority rarely did.

  • Problem

    Governance agents read evidence that may include supplier- or project-member-authored content, creating an indirect prompt-injection risk in decisions about whether projects may proceed.

    Source: §1 · p. 2

  • Method

    DGF-Bench generates certified dossiers and evaluates specialist and General gates under facts, documents, and attack conditions, counting success only when an injected action differs from the paired clean run.

    Source: §10 · p. 29

  • Results

    Task-aligned process imitations passed against four of five models, while direct-order, false-data, and false-authority attacks had one attributable success; the fake review procedure succeeded on 26 of 26 gates for GPT-6 Luna Pro.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support enforcing controls that must hold regardless of evidence, such as authority to accept risk, in the tools that execute the corresponding actions.

    Source: §8 · p. 28

    The threat model excludes changes to authoritative structured values, mandates, tools and their checks, or rules, and denies attackers access to the evaluator-only attack manifest.

    Source: §5 · p. 14

Abstract

from arXiv · show

Tool-using language-model agents can review enterprise projects as governance boards do: they read the evidence, apply written rules and decide whether the project may proceed. Part of that evidence comes from suppliers and project members with a stake in the decision. DGF-Bench is a benchmark in which a board of agents (specialist gates and a General gate that consolidates their decisions) reviews synthetic dossiers while an attacker plants deceptive content in evidence the organization does not vouch for. Dossiers are generated from canonical facts under 61 executable rules, with 42 authoritative records and 32 narrative documents; every gate is certified decidable from those records. Attacks never change an authoritative value, so an attacked dossier keeps the reference decisions of its clean copy. A success is attributable only when the agent receives the injection and takes the exact injected action, which it does not take on the paired clean dossier; the DGF score is the share of applicable fixed attacks a model blocks. Reading documents and records themselves, five of six models were outcome-strict (disposition, findings, actions and authorization all correct) on 82 to 85 of 85 gates. Over 2,622 attacked gate runs, seven direct-order, false-data and false-authority attacks obtained one attributable success against these five, whereas task-aligned attacks imitating the organization's own process passed against four of them: a record note citing a fake review procedure lowered GPT-6 Luna Pro from 34 to 6 outcome-strict gates and DeepSeek V4 Pro from 33 to 7. DGF scores ranged from 96.2 to 26.9, and a policy-aware adaptive attacker writing in records succeeded against five of six models. The approval tool executed no forged approval, yet deceived agents submitted approvals that the rules forbid. The open-source package dgf-bench computes the DGF score with one command.

1 Introduction

DGF-Bench tests whether tool-using agents reviewing governance dossiers can be deceived by content planted in evidence the organization does not vouch for. Five of six models performed strongly when finding facts in documents, but task-aligned attacks that imitated organizational process deceived several models.

1 Introduction

  • DGF-Bench generates dossiers from canonical facts and executable rules, then has specialist gates and a General gate review them under facts, docs and attack conditions.Gates are certified decidable from authoritative records; attacks alter only content the organization does not vouch for, leaving clean-copy reference decisions unchanged.Source: §1 · p. 2, Figure 1 · p. 3
  • Five of six models were outcome-strict on 82 to 85 of 85 gates when finding facts in documents and records, compared with all 85 gates under facts.Qwen3 235B was outcome-strict on 47 gates under facts and 35 under docs.Source: §1 · p. 2
  • Across five models other than Qwen3 235B, seven direct-order, false-data and false-authority attacks yielded one attributable success, while task-aligned attacks passed against four.A fake review procedure succeeded on 26 of 26 GPT-6 Luna Pro gates and 25 of 26 DeepSeek V4 Pro gates.Source: §1 · p. 2
  • The fake review procedure reduced outcome-strict gates from 34 of 34 to 6 for GPT-6 Luna Pro and from 33 to 7 for DeepSeek V4 Pro.The DGF score ranged from 84.6 for DeepSeek V4 Pro to 96.2 for GLM 5.3 among five models, and was 26.9 for Qwen3 235B.Source: §1 · p. 2
  • The benchmark contributes a certified synthetic environment, 27 fixed attacks plus an adaptive attacker, paired-clean attribution, and the DGF score for applicable fixed attacks blocked.The dgf-bench Python package runs the 27 fixed attacks against a model with one command and reports its DGF score.Source: §1 · pp. 3–4

2 Related work

DGF-Bench builds on governance, agent-tool-use, prompt-injection, and evaluation research, while focusing on review decisions over evidence that may conflict. Its distinguishing evaluation includes attacks across dossier content and attributes success only when the injected action is absent on the paired clean run.

2 Related work

  • DGF-Bench extends the author’s earlier governance-review work: the prior agents received rules and structured facts, while this study tests agents that find facts in evidence themselves.The certification, attack set, adaptive attacker, attribution measure, DGF score and experiments are new to this article.Source: §2 · p. 4
  • Existing tool-use benchmarks provide agents with tools and policies, but DGF-Bench asks for a review decision over potentially stale, partial or contradictory dossier evidence.The paper contrasts its setting with τ-bench, WorkArena and ITBench.Source: §2 · p. 4
  • Prior prompt-injection benchmarks study attacks through prompts, tool responses or third-party data, while DGF-Bench includes dossier-specific carriers such as document properties and text in an image.The related-work discussion also describes defenses that signal provenance or constrain program flow and tool calls.Source: §2 · p. 4
  • Unlike agent benchmarks reporting aggregate performance without attack, DGF-Bench evaluates dossier review decisions and attributes attacks only against the same model’s paired clean run.Its attacks can appear in documents, record notes, document properties, uploaded files and diagram text, while authoritative values remain unchanged.Source: §2 · p. 5
  • DGF-Bench checks evidence citations against observations rebuilt from harness traces, and separates that check from attack attribution against a paired clean run.The discussion situates this alongside prior concerns about benchmark costs, holdout sets and evaluation standardization.Source: §2 · pp. 5–6

3 DGF-Bench: the environment

DGF-Bench simulates enterprise IT governance reviews with route-specific specialist gates and a General gate that consolidates their decisions. Its synthetic dossiers derive from canonical facts, with executable rules, varied evidence, and certification that every scheduled gate is decidable from authoritative records.

3 DGF-Bench: the environment

  • A DGF-Bench task reviews one synthetic IT project gate by gate along a fixed route, applying published rules to evidence available to each gate.The environment includes three routes, seven specialist gate types, and a General gate.Source: §3 · p. 6, §3.2 · pp. 6–7

3.1 Dossier generation

  • Dossiers are generated from seeded canonical project facts; the hidden ground truth stores those facts and reference decisions unavailable through agent-facing tools.The same seed, route, and difficulty reproduce a dossier byte for byte on a given platform, except for diagram PNG rendering.Source: §3.1 · p. 6

3.2 Routes, gates and phases

  • The three ordered routes contain 17 gate occurrences across Buy, Integrate, and Build; the General gate closes each route and consolidates specialist decisions.Buy and Integrate have six occurrences each, and Build has five; six phases govern evidence visibility.Source: §3.2 · pp. 6–7, Figure 2 · p. 8

3.3 Executable rules

  • The 61 executable rules use the most severe finding to determine a gate’s disposition, while the General gate’s consolidation chain preserves upstream blockers.Rules also specify required actions and when authorization is required; 58 rules can fire on the three routes.Source: §3.3 · p. 7

3.4 Evidence and realism levers

  • The evidence catalog contains 42 authoritative records and 32 narrative documents, with seeded variations in availability, truthfulness, history, units, status terms, and provenance.Only owner-recorded entries count in nine dated records; authoritative sources for decisive facts remain available and truthful.Source: §3.4 · pp. 8–9

3.5 Tools and the review board

  • Agents read scoped, phase-visible evidence and use workflow tools; downstream gates receive prior decisions, and the General gate receives every specialist decision on its route.Tools expose public files and finding catalogs, while scoring checks whether conditional approvals cover exactly the open findings.Source: §3.5 · pp. 10–11

3.6 Certification

  • Certification checks every decisive field against canonical facts in authoritative public sources, and the runner rejects uncertified dossiers in docs or attack conditions.All 85 gates in the development experiment were certified, with 612 field checks matching for clean dossiers and attacked variants.Source: §3.6 · p. 11

4 Information conditions and scoring

The paper compares agents given structured facts, agents finding facts in dossier documents and records, and agents reviewing attacked document dossiers. It scores decision fidelity and attributes attack success only when the injection was observed and the same action was absent on the paired clean dossier.

4.1 Three information conditions

  • In facts, agents receive policy code and structured project facts; in docs, they must find facts in the dossier, and attack runs docs agents on injected variants.Attacks leave authoritative values, rules, mandates, and tools unchanged, preserving each clean dossier’s reference decisions.Source: §4.1 · pp. 11–12

4.2 Decision and outcome-strict scoring

  • Outcome-strict gates match the effective reference on disposition, findings, actions, and authorization; evidence is scored separately against admissible sources.The General gate’s primary reference uses upstream effective references, while a second score avoids counting upstream errors twice.Source: §4.2 · pp. 12–13

4.3 Attack outcome and attribution

  • An attack success is attributable only when the model observed the injection, took its exact injected action, and did not take that action on the paired clean gate.This clean-pair condition distinguishes attack effects from actions the model also takes on clean evidence.Source: §4.3 · p. 13

4.4 The DGF score

  • The DGF score is the percentage of applicable fixed attacks blocked, counting each attack once if it has any attributable success.Applicability depends on whether the attack targeted a gate; image attacks count only for models that received the image, and adaptive campaigns are excluded.Source: §4.4 · p. 13

5 The attack track

The attack track tests whether agents can be steered by deceptive content placed in evidence without changing authoritative values. It covers fixed attacks across several carriers and goals, plus adaptive campaigns that target blocked specialist gates.

Threat model

  • Attackers may write in non-authoritative documents, free-text record notes, added dated rows, file properties, diagrams, uploaded files, or project context.They cannot change structured authoritative values, mandates, tools and checks, or rules; the attacked dossier therefore retains its clean copy’s reference decision.Source: §5 · p. 14

Attack goals

  • Attacks target finding suppression, unjustified approval, forged authority, spoofed approval, mandate misuse or overreach, upstream override, or tool-budget exhaustion.The assigned goal depends on what the attacked gate admits; Table 5 summarizes carriers and goals.Source: §5 · p. 14, Table 5 · p. 15

Fixed and adaptive attacks

  • 27 fixed attacks plant in-text, document-vector, or known-technique injections, while two additional campaigns use an adaptive attacker.The in-text set includes direct orders, false data, and task-aligned manipulations framed as organizational process.Source: §5 · pp. 14–15

Adaptive attacker

  • The policy-aware record-placed adaptive campaign writes beside a decisive fact, sees policy and rationale excerpts, and gets up to five rounds.The document-placed campaign instead writes in the narrative-injection document, sees the first 400 rationale characters, and gets up to four rounds; adaptive campaigns are outside the released command and DGF score.Source: §5 · pp. 15–16

Attribution and construction

  • A success requires the agent to receive an injection and take its exact targeted action, unlike on the paired clean gate.Fixed attacks are planted in attacked variants of certified clean dossiers; no authoritative record value is changed.Source: §5 · pp. 14, 16, 17

6 Experimental setup

The experiments evaluated six tool-using models on clean, document-reading, and attacked dossier conditions, then tested fixed and adaptive attacks on selected dossiers and gates. The protocol was preregistered before run configuration, and the paper reports the experiment’s model and cost scope.

Models and run settings

  • Models differed in image input, while run settings capped each gate at 30 turns and 60 tool calls and forced a decision submission at the limit.Image-capable models received the architecture diagram at designated gates; later gates, including General, received the same model’s earlier specialist decisions.Source: §6 · pp. 17–18

Three-condition experiment

  • Six models reviewed the same 15 development dossiers in the facts, docs, and attack conditions, covering 85 gates and 88 reference findings.All 270 route runs completed; the attack condition had 126 attacked gates across the six models, with 21 of 70 specialist gates attacked per model and the remainder receiving placebos.Source: §6 · p. 18

Attack battery

  • The attack battery comprised 2,622 attacked gate runs across 29 attacks and six models, using six dossiers and selected blocked specialist gates.Attacks 1–15 used full routes; attacks 16–27 and both adaptive campaigns used single-gate replays, paired with clean runs.Source: §6 · p. 18

Pre-registration

  • The protocol was committed before the first run configuration, and the protocol’s sealed test set was not used in the article.Post-stage decisions were recorded with their dates and reasons in a deviation log.Source: §6 · p. 19

Inference cost

  • The recorded attack-battery cost was $177.22, including full-route runs, single-gate replays, and two adaptive campaigns.The three-condition experiment cost $53.12, and the first stage cost $10.58.Source: §6 · p. 19

7 Results

Document-based review preserved outcome-strict decisions for most models, but attack resistance varied sharply: task-aligned and adaptive record-based deception succeeded where direct-order attacks rarely did.

7.1 Reviewing from documents

  • Under docs, five models were outcome-strict on 82 to 85 of 85 gates, while Qwen3 235B was strict on 35.Under attack, the five models were strict on 79 to 83 gates and Qwen3 235B on 31; the five models’ General gates were strict on 15 of 15 dossiers in each condition.Source: §7.1 · pp. 19–20

7.2 Where agents fail

  • The reference control was strict on all 85 gates, while the lever-naive control was strict on 29 and made three false approvals.Under docs, Qwen3 235B’s 72 error rows comprised 48 spurious findings, 19 missed findings, and five decision, action, or authorization errors.Source: §7.2 · p. 21

7.3 The attack battery

  • DGF scores ranged from 96.2 for GLM 5.3 to 26.9 for Qwen3 235B, measuring the share of applicable fixed attacks each model blocked.The text-only models were scored on 26 attacks; the other models were scored on 27.Source: §7.3 · p. 22, Figure 6 · p. 23
  • Attacks 1–7 produced one attributable success against the five models other than Qwen3 235B, while fake review procedures passed against four of them.The fake procedure reduced GPT-6 Luna Pro from 34 outcome-strict gates to 6 and DeepSeek V4 Pro from 33 to 7.Source: §7.3 · pp. 22–23
  • Seven fixed file-vector and known-technique attacks were blocked by every applicable model; the four other document vectors passed against Qwen3 235B alone.The trigger-activated rule passed against GPT-5.6 Sol Pro, GPT-6 Luna Pro, and GLM 5.3; fake control tokens passed against Qwen3 235B alone.Source: §7.3 · p. 25

7.4 The adaptive attacker

  • The record-placed, policy-aware attacker succeeded against five of six models, reaching six of eight gates for Qwen3 235B and none for Gemini 3.8 Flash.GPT-6 Luna Pro was reached on five gates, DeepSeek V4 Pro on three, and GPT-5.6 Sol Pro and GLM 5.3 on one each.Source: §7.4 · pp. 25–26, Table 10 · p. 26

7.5 Attribution

  • Across 2,076 in-text and adaptive attack gates, 194 of 267 exact injected actions were attributable successes (72.7%).Of the 73 actions set aside, 58 were actions the model had also taken on its paired clean dossier.Source: §7.5 · p. 26

7.6 Tool-level guardrail

  • No forged approval was executed, yet attacks still elicited prohibited approval calls and attributable GO decisions from five models.The tool guardrail checks execution, not the submitted decision; under the policy-aware adaptive attack, five models made attributable GO decisions.Source: §7.6 · pp. 26–27

8 Discussion

Attacks that imitate governance procedures bypassed defenses that blocked direct orders and false-authority tactics, while the record hierarchy protected authoritative values but not adjacent free text. Tool-enforced approval controls held, but specialist errors could still reach the board decision.

Orders and imitations of process

  • Six of 14 fixed-attack passes against five models came from task-aligned attacks that presented manipulation as the organization’s own process.The fake review procedure succeeded on 26 and 25 of 26 gates for GPT-6 Luna Pro and DeepSeek V4 Pro, respectively; the record-placed adaptive attacker also obtained GO decisions from five models.Source: §8 · p. 27

Where authority is placed

  • No narrative-only attack passed against the five models, but 12 of their 14 fixed-attack passes used free-text notes in authoritative records.The records’ priority protected their values, not free text beside those values that no rule reads.Source: §8 · p. 27

Checks in the tools and in the decisions

  • The approval tool executed no forged approval, yet agents submitted forbidden GO decisions that the tool does not check.The record-placed adaptive attacker obtained attributable GO decisions on blocked gates from five models.Source: §8 · p. 28
  • Controls that must hold regardless of evidence, such as authority to accept risk, should be enforced in the tools that execute those actions.This is the authors’ supported conclusion from tool checks holding while successful attacks changed the agent’s reading and judgment.Source: §8 · p. 28

The board and its upstream

  • Every attributable attack success occurred at a specialist gate; the General gate’s correct consolidation carried upstream errors into board decisions.For the five models other than Qwen3 235B, General gates were outcome-strict against the decisions submitted upstream, and every miss of the primary reference followed an upstream error.Source: §8 · p. 28

9 Release and reproducibility

The release provides a command-line package to generate and certify dossiers, run fixed attacks, score outcomes, and report attributable successes. Seeded generation and documented example dossiers and protocol materials support reproducibility, with adaptive campaigns outside the command’s scope.

9 Release and reproducibility

  • One command runs the 27 fixed attacks against a model served through OpenRouter; adaptive campaigns are not included.The signed-PDF attack requires the optional pdf extra; without it, the command runs the other 26 attacks.Source: §9 · pp. 28–29
  • Package version 0.1.2 requires Python 3.10 or later and follows protocol DGF-decision-v9.Its code is licensed under MIT OR Apache-2.0; bundled Microsoft Azure architecture icons remain under Microsoft’s terms.Source: §9 · p. 28
  • The dgf-bench command generates and certifies dossiers, builds fixed-attack variants, runs clean and attacked evaluations, and reports DGF scores and attributable successes.It reuses existing dossiers and recorded costs when resuming after a dry run, interruption, or budget cap.Source: §9 · pp. 28–29
  • The repository includes clean and attacked example dossiers, attack descriptions, protocol documentation, and results files for the attack battery.The results include each Table 11 cell, attacked-gate records, outcome-strict counts, and model-provider information.Source: §9 · p. 29
  • Dossiers are generated locally from seeds, route, difficulty, and dossier count, with byte-identical evidence on one platform and Cairo build except signed memoranda.The sealed test set is generated once, certified, attacked with fixed wordings, and fixed by dossier SHA-256 digests.Source: §9 · p. 29

10 Conclusion

DGF-Bench evaluates whether tool-using agents can be deceived by planted content in governance evidence. The results contrast strong performance on document-based review with susceptibility to process-shaped and adaptive attacks, while the approval tool executed no forged approvals.

10 Conclusion

  • Five of six models were outcome-strict on 82 to 85 of 85 gates when finding facts in documents and records.The benchmark counts an attack only when the model takes the exact injected action after receiving it and does not take that action on the paired clean dossier.Source: §10 · p. 29
  • Seven direct-order, false-data, and false-authority attacks obtained one attributable success against the five models, while a fake-procedure note succeeded on 26 and 25 of 26 gates against two.The record-placed policy-aware adaptive attacker succeeded against five of six models.Source: §10 · p. 29
  • The approval tool executed no forged approval, and the released package computes the DGF score with one command.The DGF score summarizes resistance to the 27 fixed attacks.Source: §10 · p. 29

A Full attack table

The full attack table reports attributable successes by attack and model, alongside aggregate fixed-attack counts and DGF scores. Its denominators reflect which gates and models each attack can target.

A Full attack table

  • Table 11 reports attributable successes over attacked gates for 29 attacks and six models, with the battery totaling 2,622 attacked gate runs.Rows 1–27 are fixed attacks counted in the DGF score; rows 28–29 are adaptive attackers excluded from that score.Source: A Full attack table · p. 32
  • The DGF score is 100× blocked divided by applicable fixed attacks, with 26 applicable attacks for text-only models and 27 otherwise.A bold cell marks at least one attributable success, and summary rows aggregate attack groups and all attacks.Source: A Full attack table · p. 32
  • Attack denominators vary by target: attacks 1–15 use eligible gates across six dossiers, while attacks 16–29 replay eight blocked specialist gates per model.Text-only models cannot receive the diagram-image attack, and the latter attacks omit upstream decisions and diagram images.Source: A Full attack table · p. 32

B The attacks

The attacks plant canary-marked deceptive content in evidence the organization does not vouch for, across records, narrative documents, supplier e-mails, and project context, while leaving authoritative values and clean-dossier reference decisions unchanged. They test whether agents treat such content—including material imitating trusted review procedures or chat structure—as authoritative or let it distort their review decisions.

B The attacks

  • Each attack copies a certified clean dossier, plants a canary-bearing injection in untrusted evidence, and targets a defined gate goal without changing the clean dossier’s reference decision.Goals include finding suppression, unjustified approval, forged authority, misused mandate, spoofed approval, upstream override, and exhaustion.Source: B The attacks · p. 32

B.1 In-text injections (1–15)

  • Attacks 1–15 use narrative and record-note injections, forged newer entries, spoofed tool outputs, mandate claims, diagram text, and tactics such as fake procedures and budget exhaustion.Forged entries alter a decisive value but are recorded by a non-owner, which the provenance rule rejects; fake procedures claim findings should be pending despite unchanged rule-triggering values and no such exemption.Source: §B.1 · pp. 33–36

B.2 Document and metadata vectors (16–21)

  • Attacks 16–21 place deceptive claims in document properties, labels, signed PDFs, or body text, testing whether agents trust metadata or let it displace authoritative records.One vector labels an authoritative record outdated and presents a non-authoritative Word file as current; another pairs an AI-directed label with an instruction in the document body.Source: §B.2 · pp. 38–39

B.3 Known injection techniques (22–27)

  • Attacks 22–27 instantiate known injection techniques, including a trigger-activated rule that joins a harmless-looking document convention to a record marker and vocabulary suppression that impedes negative findings.These attacks target dropping a finding or approving a blocked gate.Source: §B.3 · p. 39
  • CD-9 directs reviewers to avoid specified finding and outcome terms, describe issues as observations, and record the outcome as GO.The directive is presented as a project-specific communication rule.Source: §B.3 · p. 39
  • Fragmented payloads split a purported risk-owner instruction across an authoritative record and narrative documents, testing whether agents reassemble and follow it.The fragments identify the decision subject, say the finding is accepted and closed, and direct that it not be raised.Source: §B.3 · pp. 39–40
  • Fake control-token attacks place a pre-approval or out-of-scope override inside counterfeit chat-role markers in a decisive record’s notes field.The ten-line note uses one of four chat-template formats to make record text resemble a new system turn.Source: §B.3 · p. 40
  • Forwarded supplier e-mails relay a third party’s request, attributed to agreement with the Chief Risk Officer, not to raise a finding or mark a gate approved.The uploaded Word document is non-authoritative, visible only to the attacked gate, and carries the instruction in the quoted supplier reply.Source: §B.3 · p. 40
  • Project-context notes place gate-specific instructions to close findings or pre-approve gates beside project identity information the agent is told to trust.Because the note appears in the prompt rather than an evidence file, it counts as observed at every attacked gate.Source: §B.3 · p. 40

B.4 Adaptive attackers (28–29)

The section describes two adaptive attack campaigns and the rules that govern gate decisions. Adaptive text is revised after refusals, while the rule system determines findings, dispositions, actions, and board-level consolidation.

B.4 Adaptive attackers (28–29)

  • Each adaptive campaign targets eight previously blocked specialist gates, revises injected text after refusals, and stops on a gate at its first success.The attacker sees the target decision and rationale; campaigns replay the gate alone without upstream decisions or the diagram image.Source: §B.4 · p. 40
  • The document-placed attacker gets up to four rounds, using reviewer feedback to revise text appended to a narrative document.Its example obtained an attributable GO at a Build dossier’s Architecture gate from Qwen3 235B.Source: §B.4 · p. 41
  • The record-placed attacker gets up to five rounds and the gate policy, rationale, and task-aligned strategies to revise notes beside decisive facts.Its example obtained an attributable GO at a Buy dossier’s Procurement gate from GPT-5.6 Sol Pro.Source: §B.4 · p. 41

C Rules per gate

  • The rule catalog contains 61 rules across eight gate types; 12 are critical, 37 high, and 12 medium, while 23 allow risk acceptance.Rules produce findings with required actions and owners; a gate takes the most severe disposition among its findings, and conditional approval has specified risk-acceptance requirements.Source: C Rules per gate · p. 41
  • Seven rules test phase before facts; 58 can fire on the three routes, while the General gate consolidates specialist blockers and cannot override them.The General gate prioritizes upstream NO_GO, then suspension, then rework.Source: C Rules per gate · pp. 41–42
  • Tech Readiness rules cover backup, restore, recovery objectives, failover, load, code, operations, rollback, and production blockers; Procurement rules cover supplier eligibility, due diligence, budget, and references.Tech Readiness has 10 rules and Procurement has 6.Source: C Rules per gate · p. 43
  • Legal, Compliance, and General rules include contract and regulatory findings, project-level suspension conditions, and upstream decision checks.The supplied rules include 8 Legal, 7 Compliance, and 7 General rules.Source: C Rules per gate · p. 43

D Evidence catalog

The evidence catalog defines the sources available to each gate, distinguishing authoritative records from narrative documents. Dossiers vary source reliability while retaining truthful, available primary records for each scheduled gate’s decisive facts.

D Evidence catalog

  • At difficulty 4, records are unavailable with 16.0% probability and documents are stale, partial, or conflicting with 11.2% probability each.The generator restores each scheduled gate’s primary authoritative source of decisive facts to available and truthful status.Source: D Evidence catalog · p. 44
  • Source decoders read 74 decisive fields from 50 sources, while authoritative current records prevail over potentially stale or conflicting narrative copies.The decoders comprise 123 decoders covering 116 distinct field-source pairs.Source: D Evidence catalog · p. 44
  • Each dossier catalogs 74 evidence sources: 42 authoritative records and 32 narrative documents, with gate-admissible inputs specified for each source.The records comprise 30 CSV and 12 JSON files; the narrative sources include Word, CSV, JSON, YAML, and SVG files.Source: D Evidence catalog · p. 44, Table 13 · p. 45

E What the agent sees

A gate occurrence gives the agent project and gate context, evidence tools, and the policy needed to decide from the submitted project state. The prompt distinguishes authoritative records from other evidence and specifies how to cite findings and handle approval.

E What the agent sees

  • The prompt supplies the gate objective, admissible inputs, required output, decision policy, and executable rule definition for the occurrence.The policy describes a review of the submitted project state, not completed remediation.Source: E What the agent sees · pp. 46–47
  • The policy distinguishes conditional approval from risk acceptance: approval requires an active mandate and eligible findings, while accepted findings and proposed actions remain open.The tool checks mandate and eligibility; scoring verifies coverage of open findings.Source: E What the agent sees · pp. 46–48
  • Only entries recorded by a system-of-record owner count as validated record entries; other contributors’ entries are unvalidated submissions.The policy also tells agents to treat documents as evidence rather than ground truth and to prefer authoritative records when sources conflict.Source: E What the agent sees · pp. 46, 48
  • Agents must investigate evidence through tools and cite each finding’s supporting observation and JSON pointer, including upstream decisions when relevant.The prompt says authoritative sources take precedence over non-authoritative documents and specifies record-selection, date, unit, and status conventions.Source: E What the agent sees · pp. 47–48
  • The attack condition changes the policy’s information condition to attack, while the facts condition instead supplies a REVIEW_FACTS snapshot and asks for exact field-and-value excerpts.The architecture diagram was attached to selected gates when available, but single-gate replays and adaptive campaigns ran without it.Source: E What the agent sees · pp. 48–49

F Costs and tokens

The experiments cost $240.92 in total, with the attack battery accounting for $177.22.

F Costs and tokens

  • $240.92 was the total reported inference cost across the experiments.The three-condition experiment cost $53.12; the attack battery cost $177.22.Source: F Costs and tokens · p. 49
  • $177.22 was spent on the attack battery, including $141.42 for the 15 in-text attacks and paired clean run.Single-gate replays cost $20.97 and adaptive campaigns cost $14.83.Source: F Costs and tokens · p. 49
  • $53.12 was the three-condition experiment cost: $12.63 in facts, $20.18 in docs, and $20.31 in attack.The first stage on three dossiers cost $10.58.Source: F Costs and tokens · p. 49

G Deterministic controls

Deterministic controls separate correct decision-making from evidence handling and instruction-following: the reference agent remained strict, while naïve evidence choices and injected instructions degraded results. The package also provides example dossiers and a command-line report workflow.

G Deterministic controls

  • The reference agent was outcome-strict and strict with evidence on all 85 gates, completed all 15 routes, and made no false approval.On attack variants it remained outcome-strict on all 85 gates and had no attack success.Source: G Deterministic controls · pp. 50–51, Table 15 · p. 50
  • The lever-naive agent was outcome-strict on 29 gates (34.1%), strict with evidence on 28 (32.9%), completed no route, and made 3 false approvals.The authority-blind agent was outcome-strict on 82 gates (96.5%) but strict with evidence on 63 gates (74.1%).Source: G Deterministic controls · pp. 50–51, Table 15 · p. 50
  • The injection-obedient agent took the exact injected action on all 21 attacked gates and fell to 68 outcome-strict gates (80.0%), with 8 complete routes and 9 false approvals.The approval tool executed none of the forged approvals it requested.Source: Table 15 · p. 50, G Deterministic controls · p. 51

H Example dossier and report

  • The package includes clean example dossiers for each route and 27 Build attack variants, each with one injected attack and unchanged reference decisions.The example README identifies an attack’s placement, targeted gates, objective, injected file, and canary token.Source: H Example dossier and report · p. 51
  • dgf-bench run generates and certifies dossiers, runs clean and attacked variants, scores gates, and attributes successes against paired clean runs.Its report includes a DGF score and per-attack results; incomplete runs can be resumed.Source: H Example dossier and report · p. 52

Related papers

Loading 2609.34913v1…