Task Success Is Not Enough: Side-effect-Aware Evaluation of Tool-Using Language Model Agents

1 Introduction

• Collateral damage (CD) is a write outside the user’s authorization boundary, which can occur even when an agent completes the requested task.Examples include choosing an ambiguous target or expanding the request to linked objects or unintended recipients.Source: §1 · pp. 1–2
• Task-success metrics remain the usual evaluation lens even as agents gain the ability to write to shared state.Prior safety work on malicious requests and prompt-injection attacks leaves open how often agents over-act under ordinary benign instructions.Source: §1 · p. 2
• SABENCH crosses four state structures with four trap families because collateral-damage risk depends on both state and instruction wording.The benchmark includes independent, dependent, externally reachable, and cross-tool states.Source: §1 · p. 2
• The paper formalizes CD, releases a 201-task benchmark and deterministic oracle, and evaluates 11 frontier LLMs on 2,211 model–task pairs.The reported overall CD rate is 15.2%, with strong variation by model, trap family, and state structure.Source: §1 · p. 2

2 Problem Formulation

• An evaluation instance combines an instruction, initial state, fixed tool schema, authorized writes, and post-execution goal conditions.The agent produces a tool-call trace and final response from the instruction, state, and schema.Source: §2 · p. 2
• CD is measured from successful writes in the trace that fall outside the explicit and mechanically augmented authorization sets.Writes are normalized by tool, action, target entity, modified fields, and whether they reach an external party.Source: §2 · pp. 2–3
• For ambiguous tasks, any successful write before clarification counts as an ambiguity violation.The task has no explicit authorized writes because the safe behavior is clarification rather than commitment.Source: §2 · p. 3
• The oracle replays writes to check goal satisfaction separately from authorization compliance and labels whether unauthorized writes are externally visible.The example distinguishes canceling the requested occurrence from also canceling its series, which is CD.Source: §2 · p. 3, Figure 1 · p. 3

3 The SABENCH Benchmark

• SABENCH contains 201 tasks from 47 templates arranged as a 4 × 4 matrix of state structure and trap family across three tool domains.All 16 matrix cells are populated, with task counts varying because templates are instantiated at different rates.Source: §3 · p. 3, Table 2 · p. 4
• The state structures distinguish isolated targets, dependent objects such as series or threads, externally reachable recipients, and semantically linked entities across tools.Cross-tool tasks place the authorized target in one tool and related entities in others.Source: §3 · pp. 3–4
• The four trap families are represented in Table 1 by a user request, a risky state cue, and an unauthorized write marked as CD.Each trap is mechanically defined for oracle recognition without natural-language inspection.Source: Table 1 · p. 4, §3 · p. 4
• Each template fixes the initial state, authorization set, and goal conditions, then yields four or five tasks varying entities, state contents, and wording.Templates are the resampling unit for statistical inference; the labels are coverage strata, not an exhaustive ontology, and attributes can overlap.Source: §3 · p. 4
• Of the 201 tasks, 45 are ambiguous and the corpus is roughly balanced across email, calendar, and ticket tools.Source: §3 · p. 4

4 Oracle and Evaluation

• The oracle extracts successful state-mutating calls, normalizes them, replays them in order, and checks the resulting state against goal conditions.Reads, failed calls, and idempotent calls are ignored.Source: §4 · p. 4
• The benchmark comprises 47 templates and 201 tasks across three tool domains, including 45 ambiguous and 156 non-ambiguous instances.Source: Table 3 · p. 5
• Goal conditions certify the requested outcome, while the authorization set determines whether each realized write was permitted.A write absent from the goal specification is still evaluated by the authorization check.Source: §4 · p. 5

5 Experimental Setup

• 11 frontier LLMs each ran once on 201 tasks, yielding 2,211 model–task pairs under a 16-step cap.All models received the same instructions, serialized state, and tool schema; runs used temperature 0.Source: §5 · p. 5
• The evaluation reports goal, collateral-damage, goal-plus-CD, ambiguity-violation, and external-CD rates across models, traps, states, and templates.On ambiguity tasks, a write before clarification violates the stated contract.Source: §5 · p. 5
• 95% template-clustered bootstrap intervals quantify uncertainty from the composition of the 47 task templates.The analysis resamples templates 10,000 times; pairwise model comparisons jointly resample the same templates.Source: §5 · p. 5

6.1 (RQ1) Task success understates agent risk

• 15.2% of runs incur CD, and 12.8% of goal-satisfied runs still incur CD; non-ambiguous tasks also show a 4.1% rate among goal-satisfied runs.The 4.1% figure applies to the 156 non-ambiguous tasks, where the goal is non-vacuous.Source: §6.1 · p. 5

6.2 (RQ2) Model-level CD varies by 4.7× at comparable goal rates

• CD rates vary 4.7× across models at comparable goal rates; DeepSeek V3.2 and GPT-5.4 differ by 16.4 CD points despite similar goal rates.Their goal rates are 87.6% and 88.1%, respectively; Figure 2 and Table 4 show per-model outcomes.Source: §6.2 · p. 6, Table 4 · p. 6, Figure 2 · p. 6

6.3 (RQ3) CD concentrates in specific trap families and state structures

• 38.8% of ambiguity tasks incur CD, versus 6.5% for cascade harm, 7.8% for mis-targeting, and 11.0% for overreach.Non-ambiguity traps average 8.4% CD; despite an explicit instruction to clarify, models often commit to a plausible candidate.Source: §6.3 · p. 7, Figure 3 · p. 7
• CD is higher in dependent and cross-tool states than in independent and externally reachable states: 22.1% and 21.0% versus 9.4% and 11.0%.The reported difference is an association across coverage strata, not a causal effect; AB-violation rates are also highest in dependent and cross-tool states.Source: §6.3 · p. 7, Figure 3 · p. 7

7 Analysis of Failure Modes

• The 337 CD runs include five named failure patterns, with unauthorized operations often involving email replies, ticket comments, calendar cancellations, and event creation.Table 5 presents two representative ambiguity-task failures where safe behavior is clarification before writing.Source: §7 · p. 7, Table 5 · p. 8
• Ambiguity prompts action-bias: AB violation rates reach 76.1% in dependent-state tasks and 62.2% in cross-tool tasks.Many models take a plausible action instead of clarifying; stronger models do not necessarily clarify more.Source: §7 · p. 7
• Dependent structures invite scope amplification, including occurrence-versus-series and parent-versus-child confusion, as well as notifications to additional participants.Models may treat a bounded state change as permission to update linked objects or notify others.Source: §7 · p. 7
• Cross-tool over-action can trigger writes in both related tools, contributing to the 21.0% CD rate on cross-tool tasks.Ticket comments and email replies often co-occur among unauthorized writes; spontaneous multi-tool mutations are treated as predictability-boundary violations.Source: §7 · pp. 7–8
• 169 of 337 CD runs are externally visible, and external CD distinguishes models with similar total CD rates.MiniMax M2.5 has 18.4% total CD and 8.5% external CD, while Kimi K2.5 has 18.9% total CD and 12.4% external CD.Source: §7 · p. 8
• Fail+CD reaches 7.0% for Kimi K2.5 and 6.5% for both GPT-5.4-mini and MiniMax M2.5.These runs miss the goal while still mutating state incorrectly.Source: §7 · p. 8

• In SABENCH, 12.8% of successful runs cause collateral damage (CD); CD rates for dependent and cross-tool tasks are roughly twice those for independent and externally reachable tasks.Source: §8 · p. 8
• The benchmark covers authorization-sensitive side effects in office workflows, not adversarial misuse, long-horizon planning, or cross-session memory effects.Its oracle uses an explicit task specification plus a small mechanical augmentation as the authorization boundary; a human audit agreed on 92.3% of decisive candidate-write judgments.Source: §8 · p. 8
• A single aggregate success score can obscure how often models write outside the authorization boundary, even when they complete most tasks.The authors suggest using per-task authorization sets alongside task-success metrics.Source: §8 · p. 8
• A confirmation policy reduced CD by 52–71% for three models at a cost of 13–17 goal points in a 115-task prompt-level study.A structured write preview cost less goal completion but significantly reduced CD only for Qwen 3.5 Plus; goal-only evaluation would miss the violation reduction.Source: §8 · p. 9
• Trace-level diagnosis identifies the changed object and violated authorization rule, distinguishing wrong-entity selection, scope expansion, unresolved ambiguity, and unauthorized tool boundaries.The controlled state variations isolate these questions from retrieval failure and interface exploration.Source: §8 · p. 9

• SABENCH measures email, calendar, and ticket workflows; its CD rates and model rankings should not be extrapolated to other domains.Applying the design to coding agents would require path- and file-scoped authorization, diff-based write extraction, and semantic goal checks.Source: §9 · p. 9
• Authorization and ambiguity judgments are partly policy-dependent, and the benchmark tests only the first decision point rather than a full clarification dialogue.Annotators agreed that clarification was needed on 10 of 13 benchmark-ambiguous tasks; an unnecessary-clarification guard metric accompanies goal and CD measures.Source: §9 · p. 9
• Each model–task pair was run once, so reported intervals capture template composition rather than sampling variability.Source: §9 · p. 9
• The benchmark uses a conservative authorization contract: only explicitly authorized writes and small mechanical bookkeeping exceptions count as allowed.This contract supports auditable, reproducible evaluation of confirmation gates, write previews, and tool-level permission systems.Source: §9 · p. 9

10.1 Agent Benchmarks

• Earlier agent benchmarks cover broad reasoning [2] [3], web and desktop tasks [4] [5] [6] [19] [20] [21], and professional workflows [7] [11], but usually measure task completion.SABENCH separates goal satisfaction from authorization compliance.Source: §10.1 · p. 9

10.2 Stateful Tool-Use Evaluation

• Stateful benchmarks include policy-constrained database comparison in τ-bench [8], implicit dependencies in ToolSandbox [9], and unexpected-modification checks in AppWorld [10].SABENCH makes collateral damage its primary measurement target, defining it as unauthorized trace writes and varying state structure and trap family systematically.Source: §10.2 · pp. 9–10

10.3 Safety, Function Calling, and Methodology

• Safety benchmarks often target malicious tasks or prompt injection [12] [13] [14] [15] [16], while function-calling benchmarks assess structured-call correctness [33] [34].Table 6 positions SABENCH against related benchmarks on properties needed to evaluate authorization-sensitive side effects under benign instructions.Source: §10.3 · p. 10, Table 6 · p. 10

• 15.2% of runs incur CD, including 12.8% of goal-satisfied runs, across 201 office-workflow tasks and 11 frontier models.Source: §11 · p. 10
• Ambiguity has the highest CD risk, while non-ambiguous traps average 8.4% CD; dependent state and cross-tool coupling also show more frequent errors.Source: §11 · p. 10
• Agent evaluation should assess whether a trajectory stayed within the user's authorization boundary as well as whether it reached the requested goal.A benchmark recording only goal attainment cannot distinguish careful completion from success that mutates extra state.Source: §11 · p. 10

A.1 Canonical Data and Release

• The released corpus contains 201 tasks, with complete traces and oracle reports for 2,211 model–task pairs; runs use a shared setup and template-clustered inference.Each task has a common schema for its instruction, initial state, authorized writes, and goal conditions; each model–task pair was run once at temperature 0.Source: §A.1 · p. 14, §A.2 · p. 14, §A.6 · p. 16

A.2 Oracle and Runtime Details

• The deterministic oracle extracts and replays successful writes, flags unauthorized operations, and distinguishes internal from externally visible violations.Ambiguity tasks treat any successful write as a violation; a small implicit allowance covers benign bookkeeping.Source: §A.2 · p. 14, §A.5 · p. 15

A.3 Additional Aggregate Views

• Ambiguity tasks have the highest CD rate (38.8%); dependent and cross-tool states reach 22.1% and 21.0%, versus 11.0% for externally reachable and 9.4% for independent states.Of 337 CD runs, 252 also satisfy the user goal and 169 are externally visible.Source: §A.3 · p. 14

A.4 Prompt and Tool Schema

• All models receive the same prompt and function-calling schema, which directs clarification rather than guessing when a request is ambiguous.The schema exposes email, calendar, and ticket actions.Source: §A.4 · pp. 14–15

A.7 Non-Ambiguous Outcomes and Clarification Behavior

• Among non-ambiguous runs, CD occurs in 8.4%; on ambiguity tasks, 38.8% write before clarification, while 56.8% make no write and satisfy the clarification proxy.Unnecessary clarification ranges from 2.6% to 17.3% across models, so the authors recommend reporting it alongside goal and CD rates.Source: §A.7 · p. 17

A.8 Alternative Authorization Policies

• Alternative authorization policies change absolute CD rates, but the ambiguity contrast and large model differences persist under external-only scoring.Under that policy, ambiguity CD is 27.3% versus 2.0% for other traps; Qwen 3.5 Plus has 5.8× Gemini 3.1 Pro’s rate.Source: §A.8 · p. 17

A.9 Human Validation

• A blinded audit found oracle agreement on 72 of 78 candidate writes with decisive human majorities (92.3%), while human ambiguity judgments remained contestable.The audit is a construct-validity check, not a relabeling of all tasks or part of the benchmark CD rate.Source: §A.9 · p. 18

A.10 Mitigation Study

• Confirmation reduces total CD by 52–71% across three models at a cost of 13–17 goal points, while a worked example shows a goal-satisfying reply can still include an unauthorized ticket write.The confirmation and preview interventions are prompt-level policies, not tool-enforced gates; under external-only scoring, the example’s internal ticket comment is excluded.Source: §A.10 · pp. 18–19, §A.11 · p. 19