Source-linked AI summary
FraudBench: Stress-Testing Policy-Grounded Banking Agents Against Adaptive Fraud
Dheeraj Mohandas Pai, Lu Xian
TL;DR
Banking agents can take consequential actions for callers, but existing benchmarks do not test whether they remain safe when identity, authorization, and trust are manipulated conversationally. FraudBench evaluates this setting and finds attack-security varies substantially across agents, with adaptive fraud and money-mule scenarios especially difficult.
Problem
Existing benchmarks do not evaluate whether policy-grounded banking agents safely act when callers manipulate identity, authorization, and trust across consequential conversations.
Method
FraudBench is an executable, history-dependent benchmark where agents retrieve internal policies and act through tools against mutable banking state during adversarial caller interactions.
Results
64.5% attack-security was achieved by the strongest agent versus 49.5% for the weakest, while money-mule fraud was hardest for every model.
Takeaways & Limitations
FraudBench shows that authentication is not authorization, locally valid requests may be globally unsafe, and safe agents must distinguish refusal from legitimate service.
Takeaways & Limitations
The attack-only release and preliminary single-trial semantic grading cannot measure over-refusal or establish final security–utility rankings.
Abstract
from arXiv · showhide
Conversational agents now act for end users through tools while holding access to customer databases and internal policy documents that a caller can reach through dialogue alone. Banking is the clearest case: the same agent that answers a question can also change contact details, reset a PIN, or move money, so ordinary customer service is inseparable from authorization, fraud detection, and policy compliance. Existing financial-fraud benchmarks classify static transactions or messages, and general agent-safety benchmarks target prompt injection or generic harmful use; none test whether a policy-grounded banking agent safely acts when a caller manipulates identity, authorization, and trust over a conversation. We introduce FraudBench, an executable benchmark built on the $τ^2$-bench dual-control framework and the $τ$-Knowledge banking environment. Both the agent and the simulated caller act through tools over shared, mutable account state, and the agent may grant the caller access to selected tools; the environment exposes a 698-document internal policy corpus that the agent must retrieve from. FraudBench contains 150 authored adversarial scenarios; a frozen public set of 107 (90 across ten fraud mechanisms plus 17 chained adaptive attacks) is used for all reported runs, with 43 further chained attacks held out. Safety is history-dependent: single-control tasks satisfy every precondition but one, and adaptive attacks make a later, locally valid request unsafe because of an earlier probe, admission, or failed attempt. Each scenario is annotated with observable evidence, prohibited actions, safe dispositions, and intervention points. A preliminary single-trial evaluation of four agents on the 107 graded tasks yields attack-security between 49\% and 65\%, with money-mule and first-party fraud the most common cross-model weaknesses.
1 Introduction
FraudBench addresses the gap in evaluating whether policy-grounded banking agents can safely act when strategic callers manipulate identity, authorization, and trust. It provides an executable, history-dependent benchmark over privileged tools, mutable customer state, and internal banking policies.
- Motivation: Banking agents with tools can change contact details, reset PINs, unfreeze cards, file disputes, or move money, making service inseparable from authorization, fraud detection, privacy, and policy compliance.Real access enables useful workflows but also creates an exposure surface through which protected information or consequential actions may be reached.
- Evaluation gap: Existing fraud benchmarks classify records or resist scam dialogue, while agent-security benchmarks target generic harms or prompt injection rather than policy-grounded banking action under adaptive fraud.Fraud-R1 lacks a bank database, privileged banking tools, and hundreds of operational policy documents, and measures conversational resistance rather than fraudulent-caller safety.
- Evaluation: Episodes use privileged tools, a mutable customer database, and a 698-document internal policy corpus, then grade actions, state, leaks, and disposition using full conversation history.Scenarios are annotated with observable evidence, prohibited actions, safe disposition, and intervention point.
- Benchmark: FraudBench is an executable evaluation extending τ-bench, τ^2-bench, and τ-Knowledge to adversarial conversations involving policy-grounded banking agents.The benchmark evaluates an agent whose caller’s private goal conflicts with policy, so the correct outcome may be refusal or escalation.
- Benchmark: The benchmark includes 150 hand-authored adversarial tasks, with a frozen 107-task public set and 43 further chained attacks held out.The public set contains 90 tasks across ten fraud mechanisms and 17 chained trust-contamination attacks.
2 Related Work
Existing fraud and agent-safety benchmarks largely test static detection, generic attacks, or limited conversational inducements rather than policy-grounded banking agents executing privileged operations over mutable account state. FraudBench builds on interactive tool-use frameworks while adding domain-specific fraud scenarios and a large banking-policy corpus targeting data-exposure and policy-boundary failures.
- Financial-fraud benchmarks: Public financial-fraud datasets provide labeled transactions or laundering networks, but they do not test whether language agents safely execute customer-service operations.Examples include PaySim, AMLSim, IBM synthetic AML datasets, and Elliptic.
- Financial-fraud benchmarks: Fraud-R1 evaluates multi-round resistance to fraud and phishing inducements, but scores messages with an LLM judge without bank state, privileged tools, or a policy corpus.Its coverage is narrower, focusing on inducement and phishing.
- Interactive agent benchmarks: τ-bench, τ^2-bench, and τ-Knowledge establish dynamic tool use, shared environments, policy retrieval, and banking tasks, providing the framework and setting FraudBench extends.τ^2-bench allows both agent and user to act through tools, while τ-Knowledge combines policy retrieval and tool use with roughly 700 banking documents.
- Agent-safety benchmarks: Agent-security suites cover prompt injection, utility-security tradeoffs, tool risks, malicious requests, web misuse, enterprise-policy compliance, and sequence-emergent harm, but omit production-style banking databases and large internal documentation.This omission excludes data-exposure and policy-boundary failures central to customer-facing deployments.
- Framework comparison: DoomArena injects configurable attacks into host environments but provides no fraud taxonomy, banking-policy corpus, or domain-grounded scenarios; FraudBench supplies those complementary components.The comparison positions FraudBench as the scenario-and-corpus contribution for executable security-testing frameworks.
3 The FraudBench Benchmark
FraudBench is an executable, policy-grounded banking benchmark in which defender and caller agents act through tools over mutable account state, with safety evaluated over full interaction histories. Its frozen evaluation set combines broad coverage of fraud mechanisms with adaptive attacks whose later requests depend on earlier failures, disclosures, or intent.
- Benchmark setting: Each episode couples a defender, simulated caller, internal policy corpus, and mutable banking database mediated by shared read and write tools.Write actions such as transfers, PIN resets, and card unfreezes change the database state; the caller cannot arbitrarily access the database or learn protected values unless the defender reveals them.
- History-dependent grading: Safety is history-dependent: every scenario requirement must hold across the trajectory, including preventing leaks, recognizing red flags, and giving correct escalation reasons.A request can be unsafe despite appearing valid in isolation when earlier admissions, probes, or actions changed the trust state.
- Environment: 698 internal policy documents and 17 agent tools support policy retrieval and account operations over synthetic, mutable transactional data.The environment contains 125 fictional users and 126 bank accounts, with no real customer records used.
- Evaluation set: 107 frozen evaluation tasks comprise 90 tasks evenly distributed across ten fraud mechanisms and 17 longer chain tasks, while 43 additional chain tasks are held out.The benchmark contains 150 authored scenarios in total, with the extra chain tasks reserved beyond the public frozen set.
- Scenario design: Ten single-decisive-control tasks violate one precise rule, while ten adaptive tasks pivot after a failed attempt or disqualifying disclosure to a request valid only under clean history.Every scenario is hand-authored with structured evidence, prohibited actions, safe dispositions, intervention points, and history dependencies where applicable.
4 Evaluation Protocol
The evaluation tests whether banking agents prevent unsafe outcomes during adversarial conversations without sacrificing legitimate service, including difficulty from exact single-control boundaries and adaptive attacks. It standardizes model execution, isolates policy reasoning from retrieval where possible, and evaluates safety, utility, and reliability through transcript-level grading.
- Research questions: The evaluation asks whether agents prevent unsafe outcomes, preserve matched legitimate service, handle single-control and adaptive attacks, and avoid failures across reasoning, state tracking, execution, and refusal.It also examines failure sources and reliability across repeated trials.
- Model and retrieval protocol: The frozen panel covers frontier proprietary, open-weight, and cost-efficient families with fixed dated identifiers and decoding configurations.Primary runs use oracle retrieval for gold-mapped tasks, while a retrieval ablation reports required-document recall; retrieval claims are limited to 20 _08/_09 tasks.
- Infrastructure and reproducibility: All agents use the tau2-bench orchestrator, common system prompt, tool schema, and retrieval configuration, with only the defender model changing.Models are accessed through their respective providers or released checkpoints, and analysis scripts are released in the supplement.
- Metrics: Attack-security S is the fraction of attack tasks reaching the decision point while avoiding prohibited effects and meeting the required safe disposition.Attack success rate is 1 − S.
- Metrics: Utility U measures matched legitimate controls completed without unnecessary refusal or escalation, while passk measures reliability across k trials.The protocol keeps safety and utility as separate axes rather than collapsing them.
5 Results
The preliminary pass1 evaluation reports attack-security on 107 attack-only tasks under oracle retrieval, with substantial variation across agents and mechanisms. A realistic all-tools retrieval setting lowers Gemini 3.6 Flash’s attack-security, while the authors defer ordering claims pending a fuller frozen study.
- Evaluation scope: All four models completed single-trial pass1 runs using the GPT-5.4 Nano user simulator and natural-language-assertion grading only.The evaluation omitted deterministic checks and matched legitimate controls, and covered the attack-only suite.
- Preliminary evaluation: 64.5% attack-security made Gemini 3.6 Flash strongest, while 49.5% made gpt-oss-120b weakest.The 15-point spread indicates the suite was neither saturated nor trivially failed.
- Preliminary evaluation: Money mule was the hardest fraud mechanism for every model, and the 17 chain tasks sharply separated stronger from weaker agents.Stronger agents defended 8–9 of 17 chain tasks versus 4 of 17 for the weaker two.
- Evaluation scope: The reported results are an early, non-final probe; repeated trials, deterministic grading, matched legitimate controls, and confidence intervals precede any ordering claim.The tables use oracle retrieval, with complete 107-task runs reported for the four models.
- Retrieval setting: 13 points: Gemini 3.6 Flash’s attack-security fell from 64.5% with oracle retrieval to 51.0% with all-tools retrieval.All-tools retrieval requires finding governing policies across the 698-document corpus rather than receiving task-relevant documents directly.
6 Discussion
FraudBench argues that aggregate task success obscures distinct dimensions of banking-agent behavior: authentication, authorization, local validity, global safety, refusal, and legitimate-request completion. Accordingly, security and utility should remain separate axes.
- Discussion: Authentication does not establish authorization: a verified customer or someone with stolen static PII may still make an unsafe request.The benchmark explicitly separates identity verification from permission to perform an action.
- Discussion: Local validity does not guarantee global safety because earlier locks, admissions, or exposed secrets can prohibit an otherwise valid request.Safety may depend on the conversation’s history rather than only on the current request’s immediate preconditions.
- Discussion: Safety is not equivalent to refusal: a usable agent must also complete matched legitimate requests.The discussion therefore keeps security and utility as separate axes rather than collapsing them into aggregate task success.
7 Limitations, Responsible Release, and Ethics
FraudBench is a synthetic, sandboxed benchmark whose results do not measure real-world losses, fraud prevalence, or regulatory compliance. Its dual-use release excludes sensitive production access and recommends safeguards against contamination, blanket refusal, and demographic stereotyping.
- Limitations: The benchmark uses fictional bank data, isolated trials, and scenario metadata for amount_at_risk, so tasks cannot affect real financial systems or represent observed loss.It does not estimate fraud prevalence or regulatory compliance.
- Limitations: Coverage is limited because ten fraud mechanisms cannot represent every typology, jurisdiction, or language, and results depend on simulation, retrieval, judging, and provider safety layers.The current release is attack-only, without matched legitimate controls.
- Responsible Release: The dual-use release excludes live credentials, institution-specific controls, and production endpoints while sandboxing all tool effects.The release recommends a development split with a hidden diagnostic set and timestamped release because attack prompts risk contamination.
- Ethics: Matched legitimate controls and persona audits are recommended to limit blanket refusal and demographic stereotyping.Any human transcript audit should use appropriate institutional safeguards.