Source-linked AI summary
Certified Multi-Source Integrity for Structured Agent Actions
Anmol Pandey, Aditya Jain, Liang Chen, Carsten Maple, Christo Panchev
TL;DR
LLM agents assemble privileged structured actions from adversarially corruptible sources, but existing defenses do not certify coupled action integrity under shared-source corruption. The paper characterizes safe certification and builds a maximally live, field-sensitive certifier. Across its evaluations, the certifier admits no unsafe action where its preconditions hold, while corroboration remains scarce.
Problem
Existing defenses do not certify the integrity of coupled structured actions under a corruption budget that accounts for shared upstream sources.
Source: §1 · p. 2Method
Results
Takeaways & Limitations
Genuine corroboration is a minority phenomenon under the measured upper-bound proxies, so single-sourced fields require trusted anchors rather than a radius certificate.
Source: §4 · p. 6The live-model attacker was not fully adaptive, the counts are upper-bound proxies, and the guarantee depends on Section 2 assumptions.
Source: §8 · p. 8
Abstract
from arXiv · showhide
LLM agents increasingly take privileged, often irreversible structured actions, such as paying an invoice. They assemble each action from action-critical fields in documents and tool outputs that an adversary can corrupt, and indirect prompt injection can drive the model itself to extract attacker-chosen values. Current defenses gate on a source's trust label or certify free-text answer quality. None certifies the integrity of a coupled, policy-bound structured action under a corruption budget that accounts for shared upstream sources. We characterize when such an action is safely certifiable and give the maximally live safe certifier. It admits an action only when each field clears the rule its evidence structure supports: a bounded corruption radius over corruption-distinct evidence classes, counted by a minimum hitting set so that re-publishing or laundered copies cannot manufacture a quorum, deterministic reconciliation for complementary fields, and a trusted anchor where the evidence leaves a field single-sourced. We formalize two robustness notions, validate each mechanism by ablation, and measure how often the multi-source precondition holds on sanctions designations (70,966 entities) and software supply-chain provenance (450 packages). Under upper-bound proxies, genuine corroboration is a minority phenomenon in both, and naive attestation counting overstates it, since witnesses that look independent collapse to two corruption-distinct domains once shared origin is counted. Across five current models in a real agent loop, a realistic injection fools every model but one and a naive agent then executes the fraudulent action on most attacks. The certifier admits no unsafe action and recovers the correct value where corroboration permits, while action-gating and provenance baselines are broken in every world of our harness by some attack in its space.
1. Introduction
Privileged structured actions combine values from adversarially corruptible sources, and an injected source can induce a well-formed but fraudulent action. Existing defenses do not certify coupled action integrity under shared-source corruption, so the paper develops and evaluates a field-sensitive certifier.
1. Introduction
- Indirect prompt injection can make an agent extract an attacker-chosen value, risking an irreversible fraudulent action such as a payment.Action-critical fields such as payee, amount, and account are assembled from documents and tool outputs that may come from different parties.Source: §1 · p. 1
- Existing trust-label and free-text robustness defenses do not certify structured-action integrity under corruption budgets that account for shared upstream sources.Trust labels address whether data may flow to an action, while answer certification counts passages that duplicated copies can inflate.Source: §1 · pp. 1–2
- The certifier admits an action only when every field clears its evidence-appropriate rule, using corruption-distinct classes, deterministic reconciliation, or a trusted anchor.A minimum hitting set prevents republished or laundered copies from manufacturing a quorum; complementary and single-sourced fields use reconciliation and anchors, respectively.Source: §1 · p. 2
- Ablations show each mechanism is load-bearing, while measurements find multi-source corroboration scarce and naive agents execute fraudulent payments on most attacks.The paper also reports that one of five models resisted the injection and that the certifier outperformed capability-gating and policy-monitor defenses on safety and utility.Source: §1 · p. 2
- The live-model attacker is property-based rather than optimizing, and real evidence corroborates only a minority of relations.The account field relies on a trusted anchor rather than the corruption radius.Source: §1 · p. 2
2. Threat Model and Definitions
The paper models an adversary that corrupts control domains affecting structured-action evidence, then derives when safe execution is possible. Its certifier uses evidence-appropriate field rules, with guarantees conditional on trusted inputs and infrastructure assumptions.
2. Threat Model and Definitions
- Each attestation is tied to an originating control domain, and a trusted obligation skeleton fixes the transaction and policy constraints before untrusted evidence is read.The adversary can alter attestations in controlled domains but cannot forge identifiers for uncontrolled domains; the skeleton prevents evidence from steering certification toward a different object.Source: §2 · p. 3
- Corruption-distinctness counts the smallest set of control domains that intersects every attestation dependency set, rather than counting apparent witnesses.This is a minimum hitting set; for a threshold predicate, the presence-removal radius is m −1.Source: §2 · p. 3
- The certifier uses agreement or escalation for identity fields, unique-threshold admission for value fields, reconciliation for complementary fields, and trusted anchors for single-sourced fields.The threshold rule requires support of at least N −k and that the admitted value uniquely clears the threshold; a reconciled amount is bounded by authorization, not given a radius.Source: §2 · p. 4
- Exact-value tolerance requires an unchanged certified value under corruption, whereas safety with abstention permits either the correct value or refusal.Safety with abstention gives up availability under attack but can be reached with less redundancy, relevant because real multi-source coverage is scarce.Source: §2 · p. 4
- Safe execution is possible exactly when the set of budget-feasible actions is a singleton; executing that unique action and otherwise abstaining is maximally live.Feasibility is joint across fields because one corruption set must explain their disagreements; when multiple actions remain feasible, indistinguishable allowed worlds prevent safe commitment.Source: §2 · p. 4
- Agreement-or-escalate is (N −1)-safe when at least one of N corruption-distinct classes is honest, while unique-threshold admission is maximally live.For disjoint domains, exact-value certification against budget k is possible iff N > 2k, yielding radius ⌊(N −1)/2⌋.Source: §2 · p. 5
- Authenticated domain-bound voting limits an adversary controlling k domains to at most k votes per predicate, regardless of how many records or copies it emits.Counting provenance roots alone is insufficient because one compromised authority can issue multiple distinct original records.Source: §2 · p. 5
- The guarantees assume authentication, trusted anchors and join keys, and sound extraction; a shared model, prompt, canonicalizer, or provider can create common-mode faults.The extractor is a shared control domain, so source isolation does not remove systematic parser faults or malicious-provider risk.Source: §2 · p. 5
3. Mechanism Validation
A model-free harness tests whether removing individual mechanisms opens their corresponding attack families. The reported ablations isolate the join key, trusted anchor, mandatory-source rule, and claim-atomic counting as load-bearing components.
3. Mechanism Validation
- Removing the trusted join key, onboarding allowlist, or mandatory-source rule opens amount-splicing, name-matched-mule, or single-source attacks, respectively; the full certifier admits none.In the counting panel, laundering succeeds without domain-bound counting and Sybil addition succeeds without authentication.Source: §3 · pp. 5–6
- A three-source relation abstains under one corrupted source without certifying a wrong value, while a five-source relation continues certifying the correct value.The five-source case is the exact-value regime of the threshold result.Source: §3 · p. 6
4. How Often Does the Multi-Source Precondition Hold?
The paper measures how often real evidence supports multi-source certification, finding that genuine corroboration is limited and apparent witnesses can share a corruption domain. On corroborated packages, the certifier catches a registry substitution that a registry-trusting baseline accepts.
Sanctions designations
- Across 70,966 designated entities, the control-domain proxy leaves about 15% with 3+ domains and roughly three-quarters single-domain; these mapped counts upper-bound genuine corroboration.Recovering true corruption-distinct authorities would require aligning programme, measure, and legal basis.Source: §4 · p. 6
Software supply-chain provenance
- On 450 packages, the certifier certified no malicious source after a registry compromise (0 of 450), while a registry-trusting baseline accepted it every time.The two corruption-distinct witnesses support safety with abstention, not exact-value certification, under a one-corruption adversary.Source: §4 · p. 6
- Repository reputation is an eligibility signal, not a third witness to package origin; its availability is reported for legitimate packages, not as malicious-package detection.Across 307 declared repositories, 74% met the stated age, stars, and contributor bar.Source: §4 · p. 6
- 16.4% of npm and 32.3% of PyPI packages appeared to have three witnesses, but collapsing their shared GitHub root left none with three and about 90% with exactly two.The figure reports 450 npm packages and 300 PyPI packages.Source: §4 · p. 6, Figure 2 · p. 7
5. Model in the Loop
The real-agent-loop evaluation tests whether a certifier can safely gate payments when models read evidence that may contain an injection. The certifier admitted no unsafe actions, while the naive agent executed fraudulent payments on most attacks.
Evaluation setup
- The payment action comprised payee, account, and amount fields assembled from N evidence documents, read separately for certification and together for the naive baseline.Attacked episodes had k = 1 corrupted document.Source: §5 · p. 7
Results
- Across five models, the certifier admitted no unsafe action, while the naive agent executed fraudulent payments on 72–100% of attacks.Each model ran 80 episodes; the certifier’s guarantee held whether or not the model resisted the injection.Source: §5 · p. 7
- Every model but Opus 4.8 followed the injection on every corrupted source, including GPT-5.5 and Gemini 3.1 Pro.The evaluation used an embedded social-engineering injection in one of five evidence documents during attacked episodes.Source: §5 · p. 7
- With N = 3, the agreement rule turned a corrupted extraction into safe abstention; with N = 5 > 2k, the threshold rule recovered the correct value and abstained on only 0–6% of attacks.In clean episodes, false abstention was 0–12% under agreement and at most 4% under the threshold rule.Source: §5 · p. 7
6. Adaptive Attacker and Proxy Baselines
The evaluation compares the certifier with action-gating and policy-monitor baselines on multi-source actions. In the adaptive harness, the certifier withstands tested attacks that break both baselines, while corroboration also supports high utility in the live-model comparison.
6. Adaptive Attacker and Proxy Baselines
- Under the threshold rule, the certifier returns the correct value in 96 to 100% of attacked cases overall and pays 100% of novel payees.The policy monitor pays 0% of novel payees because it requires a maintained ground-truth list.Source: §6 · p. 7
- On Haiku 4.5, the certifier pairs 0% unsafe actions with 98% utility, a pattern reported across all five models.The naive agent is fully unsafe; capability gating executes nothing; PCAS is 18% unsafe at 29% utility.Source: §6 · p. 7, Figure 3 · p. 8
- Across 2,000 worlds, the adaptive attacker never forces unsafe execution against the certifier but breaks both baselines in every world.Each baseline still blocks one attack family, as shown in Table 7.Source: §6 · p. 8
- The harness baselines fail on four shared attack families: action-gating admits all but allowlist-missing destinations, while provenance-only admits all but currency swaps.Each blocks one family the other misses; the certifier blocks none of the families.Source: §6 · p. 8
- In the exact-value-radius sweep, two corroborating payee sources lead to abstention, while three or more certify the correct vendor.For k = 2, certification requires N ≥5; for k = 3, N ≥7. Honest-world false abstention remains a few percent under the threshold rule.Source: §6 · p. 8
7. Related Work
Prior defenses gate actions by input provenance or certify free-text answer quality, whereas this work certifies structured action fields using corroboration across corruption-distinct sources. Its closest methodological neighbor shares an isolate-then-aggregate pattern, but counts independent control domains rather than passages.
7. Related Work
- Information-flow and capability systems gate actions by where data came from, so decisive values from default-untrusted sources trigger blanket refusal rather than value-correctness certification.These include CaMeL [4], FIDES [5], PCAS [6], and AuthGraph [7].Source: §7 · p. 8
- RobustRAG [2] isolates retrieved passages and aggregates answers for a bounded-injection quality guarantee, while this work counts corruption-distinct domains and certifies field-typed actions.Laundered evidence can inflate passage counts but not domain-bound counts.Source: §7 · p. 8
- Unlike ARGUS [10], which traces argument-producing spans in one observation, this work counts corroborating independent sources; its electorate is inferred from shared dependencies.Domain-bound counting also prevents duplicated evidence from manufacturing a quorum, unlike provenance-root counting.Source: §7 · p. 8
8. Limitations and Conclusion
The paper bounds its claims by the scope of its attacker study, evidence proxies, assumptions, and the availability of corroboration.
8. Limitations and Conclusion
- The claim is bounded by untested fully adaptive live-model attacks, upper-bound count proxies, Section 2 assumptions, and scarce corroboration.Where corroboration exists, the certifier admitted no unsafe action even when the model followed the injected instruction.Source: §8 · p. 8
A. Proofs
The proofs establish the unique-feasible-action condition for safe execution and specialize it to threshold, agreement, and corruption-domain counting rules. They also show that field-wise composition remains sound but can be more conservative than joint certification.
A. Proofs
- Execution is safe exactly when the budget-feasible action set is a singleton, and the unique-feasible certifier has the smallest abstention set among safe certifiers.Soundness follows because two feasible actions can each correspond to a different true action under allowed corruptions; a singleton identifies the truth.Source: A. Proofs · p. 10
- For N corruption-distinct classes and budget k, the threshold rule certifies the true value when N > 2k, with radius ⌊(N − 1)/2⌋.At N ≤ 2k, two values can be feasible and the rule abstains; unanimity instead gives (N − 1)-safety with abstention.Source: A. Proofs · p. 10
- Counting votes once per authenticated control domain limits an attacker controlling at most k domains to at most k effective votes, regardless of copied records.Counting per provenance root would let one compromised domain emit enough distinct records to meet any fixed quorum.Source: A. Proofs · p. 10
- Field-wise composition is k-safe when every field admits a unique value, but it can abstain where a joint budget makes only one whole action feasible.The joint certifier's abstention set is contained in the field-wise composition's.Source: A. Proofs · p. 10
- The anchor rule places a single-sourced field outside the corruption budget; reconciliation and anchoring are not feasibility votes.The maximal-liveness result belongs to the joint unique-feasible-action rule, while the field-wise implementation is safe but may abstain more often.Source: A. Proofs · p. 11
B. Experimental Details
The experiments examine certifier behavior under adaptive attacks, increasing corruption budgets, and alternative vote identities, alongside live-model extraction tests. They also report canonicalization behavior and implementation choices for computing and maintaining corruption-distinct counts.
Mechanism ablation
- Each ablation opens exactly one attack while the full certifier stays safe under a single corrupted class.The caption identifies the reported values as attacker success rates.Source: Table 3 · p. 11
Corruption radius
- With N > k, no tested configuration certifies a wrong value; correct certification begins at N > 2k, requiring N ≥3 for k = 1, N ≥5 for k = 2, and N ≥7 for k = 3.The sweep used 4,000 adversarial configurations per cell, including coordinated votes by the k controlled classes.Source: B. Experimental Details · p. 12, Table 4 · p. 12
Vote identity
- Counting votes per authenticated control domain preserves the radius, whereas raw-attestation counting is immediately unsafe and root counting fails under multiple originals or inflated roots.Control-domain counting certifies the true value whenever N > 2k, regardless of how many original records or copies the adversary manufactures.Source: B. Experimental Details · p. 12, Table 5 · p. 12
Experimental comparisons
- Both proxy baselines remain unsafe on four shared attack families, while the neighboring-systems comparison tracks six certificate dimensions rather than ranking systems overall.Table 7 also includes six attack families: vendor substitution, a fresh mule account, an account carried across vendors, an inflated amount, a currency swap, and a fully alternate invoice.Source: Table 6 · p. 12, Table 7 · p. 13
Implementation
- The hitting-set implementation forces sole-cover domains before solving the small remainder, but greedy approximation overstates the count and radius; unknown provenance is grouped conservatively, lowering liveness rather than safety.Dependency sets are curated from published metadata and reviewed quarterly; the implementation falls back to greedy cover only when the residual universe exceeds twenty-two domains.Source: B. Experimental Details · p. 13
Live-model evaluation
- The live-model study uses isolated per-source reads for certification and a separate holistic call for the naive baseline; GPT-5.5 follows injections on every attacked episode yet proposes attacker values in 85% of agreement and 72% of threshold-rule attacks.The two rates measure different calls, and the holistic responses in those episodes parsed successfully.Source: B. Experimental Details · pp. 11, 13
Extractor independence
- Across six extraction configurations, no honest source produced the attacker’s value, no two honest sources agreed on a wrong value, and no unsafe action was certified in 654 reads per configuration.The reported proxy for coincident honest-source error is bounded below 2% per configuration; results are not pooled because errors are correlated by template and model family.Source: B. Experimental Details · p. 13
Canonicalization
- Case-and-whitespace canonicalization did not fold script, so transliterated correct payees compared unequal and reduced liveness without producing an unsafe action.Opus transliterated consistently and never abstained, while Haiku 4.5 did so inconsistently and abstained on 4%.Source: B. Experimental Details · p. 14