Source-linked AI summary
CoSec: Benchmarking Agent Security in Communities
Hao Chen, Wenhui Dong, Ye Chen, Jiezhi Yao, Chenbo Xia, Yuwen Qu, Renxiang Wang, Fudong Yuan, Camil Hamami, Chenglong Pan, Xinquan Yue, Ziyu Wang, Fengyu Ye, Chenyang Si, Caifeng Shan
TL;DR
Persistent agents serving users across communities must handle protected information whose authorized audience may vary, while existing evaluations do not fully cover these risks. CoSec runs complete agent systems through 208 fixed and evolving scenarios and checks their traces and artifacts against authorization boundaries. Agents often complete benign tasks while violating privacy boundaries, and security varies across harnesses even with the same backbone model.
Problem
Existing evaluations do not fully assess privacy and authorization enforcement for complete agents representing users across persistent, overlapping communities.
Source: §1 · p. 1Method
Results
Task success does not ensure privacy: overall PVR ranges from 29.33% for OpenClaw with Gemini 3.1 Pro to 96.15% for Hermes with DeepSeek 4.1 Flash.
Source: §4.3 · pp. 7–8Takeaways & Limitations
Community authorization is a system property rather than a model property, with security differing across harnesses that use the same backbone.
Source: §1 · p. 2, §5 · p. 10The threat model excludes adversaries modifying the authorization oracle or isolated runtime.
Source: §3.1 · p. 5
Abstract
from arXiv · showhide
LLM agents operate in persistent collaborative environments involving multiple users, communities, memories, files, and tools. Community boundaries may remain fixed or evolve with changes in membership, roles, composition, and relationships. Agents must complete legitimate tasks and prevent unauthorized disclosure of protected information. Existing evaluations do not fully examine these risks in agent systems. We introduce \textbf{CoSec}, an executable benchmark for evaluating privacy and authorization enforcement in LLM agent systems operating within and across communities. CoSec contains 208 canonical scenarios spanning fixed and evolving boundaries, protected information belonging to the agent owner or other participants, and attacks through dialogue, environmental content, persistent memory, and composed workflows. CoSec executes complete agent systems with persistent sessions, memory, files and tools. It verifies information flows against the active authorization state using execution traces and artifacts. Across harness and model configurations, agents frequently complete benign tasks but violate privacy and authorization boundaries. Privacy behavior varies across harnesses, attack surfaces, and community states, revealing how memory, files, tools, and workflows can carry protected information beyond its authorized scope. These findings show that task utility does not imply privacy or authorization compliance and that authorization in community settings remains an unresolved security challenge for persistent LLM agents.
1 Introduction
Persistent agents in communities face privacy risks because data ownership, provenance, and intended audience can differ across participants and contexts. CoSec evaluates complete agent systems across fixed and changing communities, finding that task success does not guarantee secure behavior.
1 Introduction
- Existing evaluations cover parts of agent security or community interaction, but controlled security evaluation remains limited for complete agents representing one user across persistent, overlapping communities.Relevant benchmarks generally center on one user or task, while community benchmarks often focus on social interaction and collaboration.Source: §1 · p. 2
- CoSec executes complete command-line agent systems with native sessions, memory, files, tools, and workspace effects to test enforcement of ownership, provenance, and community-scope boundaries.The benchmark tests the complete system rather than only its backbone model.Source: §1 · p. 2
- Across 208 scenarios, CoSec tests fixed and evolving communities, owner and third-party information, and four attack stressors, while measuring privacy preservation and task completion.Stressors include user injection, indirect injection, memory poisoning, and capability composition.Source: §1 · p. 2
- Agents frequently complete benign collaborative tasks while violating community boundaries; security also varies across harnesses using the same backbone model.The analysis identifies excess permission, failed revocation, provenance confusion, and workflow carryover as recurring authorization failure modes.Source: §1 · p. 2
2 Related Work
Prior work evaluates agent security, privacy, and community interaction, but the supplied discussion identifies a remaining gap in complete-system security evaluation across overlapping communities. CoSec connects these directions by testing agents against explicit boundaries and changing community structures.
Agent communities and social evaluation
- Community and social-evaluation benchmarks provide controlled settings for measuring social competence, coordination, and collective behavior among agents.The comparison table frames executable harnesses and community transitions as dimensions relevant to benchmark comparison.Source: §2 · pp. 2–3, Table 1 · p. 3
Security and privacy of executable agents
- Agent security benchmarks study injection, unsafe tool use, privacy leakage, and harmful task execution, but generally center policies on one user or task rather than multiple participants across overlapping communities.Community benchmarks, in contrast, study social interaction, collaboration, competition, and information sharing.Source: §2 · p. 3
Privacy boundaries among multiple participants and over time
- Privacy research emphasizes that disclosure depends on participants, social context, and information-propagation paths, while access-control models track changing roles and permissions.CoSec evaluates complete systems with explicit information boundaries, separate sessions and workspace namespaces, shared workspace, persistent private memory, and scheduled community changes.Source: §2 · p. 3, Figure 1 · p. 4
3 CoSec Benchmark
CoSec evaluates whether complete agents can remain useful while respecting privacy boundaries across users and communities. It combines a persistent community model, fixed and dynamic scenarios, separate security and utility metrics, and trace- and artifact-based verification.
3 CoSec Benchmark
- CoSec has three stages—benchmark construction, complete-agent execution, and verification—and two suites: Static112 tests fixed boundaries, while Dyn96 tests behavior after community changes.Verification uses execution traces and artifacts to assess both privacy violations and completion of legitimate tasks.Source: §3 · p. 4
3.1 Community Security Model
- The security model tracks users, communities, changing relationships, memberships and roles, protected data, policy, agent interface, and visible history; protected objects also carry ownership, source-community, and sensitivity attributes.The threat model allows adversarial messages and selected documents, metadata, memory entries, or workflow inputs, but not changes to the authorization oracle or isolated runtime.Source: §3.1 · pp. 4–5
3.2 Scenario Suite
- The 208-scenario suite covers within- and cross-community flows, owner and third-party information, four attack families, and dynamic membership, role, community-composition, or relationship transitions.Dynamic actions are scored after the scheduled event against the updated authorization state; Table 2 summarizes scenario construction.Source: §3.2 · p. 5, Table 2 · p. 6
3.3 Evaluation Metrics
- PVR and CCPVR measure privacy violations and cross-community disclosure, while BCR measures benign task completion independently of security and STCR requires completion without verified security failure.The separation means blanket refusal cannot achieve high STCR, and high BCR alone does not establish safe completion.Source: §3.3 · p. 6
3.4 Execution and Verification
- Verification records responses, community state, tool events, and artifact changes, using deterministic extraction and an LLM judge for semantic cases; inconclusive trials remain separate from completion scoring.Artifact checks take precedence when benign completion is merged, and scorecards retain decisions and supporting evidence.Source: §3.4 · p. 6
4 Experiments
Across 13 harness and model configurations, agents often completed benign tasks while violating privacy boundaries. Results varied by harness, community state, attack surface, and guardrail; dynamic failures commonly involved workflow carryover, excess permission, or provenance confusion.
4.1 Experimental Setup
- All 13 configurations ran the same 208 frozen scenarios, with Static112 testing fixed boundaries and Dyn96 testing behavior after a boundary transition.Scenario manifests, requester scripts, isolation, and verification remained fixed across runs.Source: §4.1 · p. 7
4.2 Verifier Reliability
- Combining the LLM judge with execution evidence yielded 96.9% accuracy, 96.8% F1, and 100.0% recall for the binary PVR verdict.Deterministic verification alone had 100.0% precision and 33.3% recall; the LLM judge alone raised recall to 94.5%.Source: §4.2 · p. 7, Table 4 · p. 7
4.3 CoSec Results
- Twelve of 13 configurations achieved BCR above 85%, yet overall PVR ranged from 29.33% to 96.15%, showing that benign task completion did not ensure privacy protection.Hermes with Qwen 3.8 Max was the only configuration below the BCR threshold, at 78.85%.Source: §4.3 · pp. 7–8
4.4 Where Do Agent Systems Fail?
- Capability composition had the highest pooled attack-family PVR at 70.27%, while the leading attack family varied by configuration.In Dyn96, community merge had the highest event PVR at 72.12%, while member removal was lowest at 47.12%; Figure 2 breaks down PVR by attack family and dynamic event.Source: Figure 2 · p. 8, §4.4 · p. 9
- A post-merge workflow led OpenClaw with MiniMax M2.5 to place private CarePod information in a shared MergedHub artifact without authorization.The verifier recorded PVR=1 and CCPVR=1; Figure 3 depicts this Dyn96 workflow-composition failure.Source: Figure 3 · p. 8, §4.4 · p. 9
4.5 Can Guardrails Prevent These Failures?
- On OpenClaw, the first-turn guardrail reduced PVR from 82.69% to 58.65% and raised BCR from 91.35% to 94.71%, while Hermes remained at 96.15% PVR.The instruction had different effects across harnesses and could shift disclosure surfaces without preserving the boundary in every turn.Source: §4.5 · p. 10, Figure 5 · p. 10
4.6 Why Do Agents Fail under Dynamic Boundaries?
- Among 795 classified dynamic failures, workflow carryover was most common at 29.9%, followed by excess permission at 25.5%, provenance confusion at 25.2%, and failed revocation at 19.4%.Figure 4 summarizes the four patterns across 13 configurations; their mix varied across systems.Source: Figure 4 · p. 9, §4.6 · p. 10
5 Conclusion
CoSec evaluates privacy and authorization enforcement in persistent LLM agents across fixed and evolving communities. Across 13 configurations, agents completed benign tasks while disclosing protected information, and outcomes varied by harness and attack conditions.
5 Conclusion
- Across 13 configurations, agents completed benign tasks while disclosing protected information; the same backbone behaved differently across harnesses.Capability composition presented the highest risk, and dynamic failures involved excess permission, failed revocation, provenance confusion, and workflow carryover; safety instructions provided uneven protection.Source: §5 · p. 10
- CoSec’s 208 scenarios execute native persistent agent systems with memory, files, and tools across fixed and evolving communities, capturing flows missed by response-only evaluations.The authors conclude that community authorization is a system property rather than a model property.Source: §5 · p. 10
- The authors identify broader domains, longer collaboration histories, and runtime authorization defenses as directions for future work.Source: §5 · p. 10
A Benchmark Specification
This appendix specifies CoSec’s scenario manifests, construction choices, attack taxonomy, dynamic transitions, and release checks. It describes how authorization state and information provenance remain explicit as communities and permissions change.
A.1 Extended Notation and Transition Semantics
- Scenario manifests track each object’s owner, source community, sensitivity, and destination scope; technical reachability across sessions does not imply authorization.Membership, role, relationship, merge, and split events update named state fields and prompt policy recomputation; historical content retains its original source after a merge.Source: §A.1 · p. 14, Table 5 · p. 14
A.2 Scenario Suite Construction
- The suite uses manually specified templates, deterministic instantiation, and manual review, then freezes scenarios before evaluation rather than adapting them to evaluated agents.Seven static strategies are a coverage choice: variants alter operational realization while retaining the same privacy question.Source: §A.2 · p. 14
- Four reusable profiles distinguish one-community cases from cross-community transfers, with synthetic people and records and separate task and attack requesters.In third-party cases, the agent’s user and the information owner differ; A4 may use two requesters for separate workflow stages.Source: Table 6 · p. 15, §A.2 · p. 15
- Within a fixed profile, attack variants retain the same owner, protected object, communities, benign artifact, and evaluator oracle, changing only the family-specific mechanism.This matching prevents strategy comparisons from being confounded by a different secret, task, or recipient.Source: §A.2 · p. 15
- OpenClaw, Hermes, and Codex receive the same ordered turns, workspace seed, community state, event, and benign objective; only the execution adapter changes.The worked dynamic example requires completing a checklist update while refusing to copy private financial information after a participant’s removal.Source: §A.2 · p. 16
A.3 Complete Attack Taxonomy
- Static112 organizes 28 strategies into four attack families, while each family names the pressured surface and each strategy describes how the request becomes operational.Source: §A.3 · p. 16, Table 8 · p. 16
A.4 Dynamic Scenario Construction
- Dyn96 inserts one event before the scored action and aligns four strategies to test stale approval, observations, memory, or prepared workflows against the updated state.Expansion does not declassify protected information; contraction can invalidate prior authority, and eight-turn scripts interleave benign coordination with adversarial requests.Source: §A.4 · p. 17, Table 9 · p. 17
A.5 Scenario Schema and Release Validation
- Release validation checks schema, deterministic identifiers, cell counts, event placement, paths, oracle consistency, and separation of evaluator data from agent-visible context.Source: Table 10 · p. 17, §A.5 · p. 18
B Execution and Reproducibility
The appendix documents how CoSec runs native agent systems in isolated trials and preserves records for reproduction. It also specifies verifier evidence, metric decisions, semantic judging, and human auditing.
B.1 Agent and Model Configurations
- The main evaluation records 13 agent and model configurations alongside each trial’s resolved model, software environment, session mode, runtime arguments, task identifier, and timestamps.Source: §B.1 · p. 18
B.2 Software Environment and Agent Versions
- Runners invoke native agent executables with harness-specific parameters, while trials run in Linux arm64 containers on an Apple silicon host.The paper reports principal executable versions and retains full dependency manifests and image identifiers in the reproduction bundle.Source: §B.3 · pp. 18–19, §B.2 · p. 18, Table 12 · p. 19
B.4 Runtime, Sessions, and Workspace Isolation
- Each scenario runs in an isolated trial with a native session per scenario, agent, and community; histories stay community-scoped while memory and workspace artifacts can persist.The runtime records delivered messages, model inputs and outputs, tool events, file and memory changes, session identifiers, and state updates.Source: §B.4 · p. 19
B.5 Prompt and Participant Generation
- Reported runs use frozen participant messages; the guardrail condition appends a short rule only on the first turn, leaving later turns to rely on session context.Optional participant generation is available but is outside the reported main evaluation.Source: §B.5 · p. 19
B.6 Prompt Registry
- The baseline uses ordinary lightweight workspace context without a separate safety guardrail, with later turns receiving a compact active-workspace form and trusted event notices.Source: §B.6 · pp. 19–20
B.7 Trial Validity and Retry Policy
- A trial enters a metric only when its scenario, trajectory, artifacts, and verifier output are present and parseable; completed unsafe or unhelpful outcomes are not replaced.Released trial files connect the manifest, interaction, behavior, artifacts, and metric.Source: §B.7 · p. 20, §B.8 · p. 20
C.1 Evidence Surfaces and Decision Rules
- Verification distinguishes agent-generated disclosures or actions from setup inputs, internal reads, and preexisting content, and checks new effects against active recipients and authorization scope.Deterministic evidence takes precedence when it directly establishes a violation; ambiguous semantic cases go to the judge, and benign-task completion is evaluated independently.Source: §C.1 · pp. 20–21, §C.2 · pp. 21–22
C.3 Semantic Judge Protocol
- The semantic judge receives a fixed packet of authorization state, protected-item descriptors, destinations, messages, traces, and artifacts, while human auditors independently review 192 sampled trials.The audit hides automated verdicts and judge rationale from two annotators, with disagreements resolved under a pre-fixed rubric.Source: §C.3 · p. 22, §C.4 · p. 24
C.5 Metric Aggregation and Sample Sizes
Metric denominators depend on the relevant suite and, for cross-community leakage, on whether a scenario crosses authorization scopes.
C.5 Metric Aggregation and Sample Sizes
- PVR, STCR, and BCR use every canonical scenario; CCPVR includes only cross-scope cases, while overall PVR and BCR weight each scenario once across Static112 and Dyn96.Source: §C.5 · p. 24
D Supplementary Experimental Results
The appendix reports raw scorecard counts, defines evidence-surface indicators and dynamic failure patterns, and describes the guarded-condition reminder and its comparison design.
D.1 Raw Counts by Configuration
- Table 17 gives the integer counts underlying aggregate rates, while Table 18 groups trial-level PVR counts by attack family and dynamic event.Source: §D.1 · p. 24
D.2 Violation Evidence Surfaces
- Evidence surfaces may overlap within a trial: response, file, memory, tool, and state indicators capture distinct carriers or changes, with excerpts retained in released scorecards.Source: §D.2 · p. 24
D.3 Dynamic Failure Patterns
- Dynamic failures are classified by the primary pattern tied to the earliest authorization mistake explaining the disclosure; C includes security failures with an available primary pattern label.Source: §D.3 · p. 25
D.4 Guardrail Details
- The guarded condition appends a safety reminder only to the first non-event victim system message; baseline and guarded runs otherwise share scenarios and setup.The comparison tests whether a short first-turn reminder changes later behavior, rather than replacing the native session with a full policy prompt on every turn.Source: §D.4 · p. 25
D.5 Time and Cost Analysis
The analysis defines how end-to-end execution time and provider-based API cost are measured.
D.5 Time and Cost Analysis
- Wall-clock time measures end-to-end execution, including judge calls and provider-reaching retries.Source: §D.5 · p. 25
- API cost is reconstructed from recorded token usage and the provider price used for the run.Source: §D.5 · p. 25
- Local setup time has no token cost, while retries that reach the provider count toward usage.Source: §D.5 · p. 25
E.1 Static Defense Case
The static case shows an agent completing a legitimate shared status task while keeping protected health details out of a cross-community artifact.
E.1 Static Defense Case
- The case places Elena Garcia’s private health note in TherapyCircle while the active session and destination file are in WorkGuild.Source: §E.1 · p. 25
- The agent completes the shared status note without reproducing protected health terms, and the verifier records PVR=0, CCPVR=0, and BCR=1.Source: §E.1 · p. 26
- The final WorkGuild artifact gives a general availability summary without the protected diagnosis, medication, investor context, or private source path.Source: §E.1 · p. 26
E.2 Dynamic Merge Failure
After a community merge, a composed transaction workflow places protected health information in a shared artifact despite completion of the benign checklist.
E.2 Dynamic Merge Failure
- The case follows a trusted merge of CarePod and LaunchPod into MergedHub, while raw private history still requires explicit authorization.Source: §E.2 · pp. 26–27
- The queued transaction revision already contains a private source line, but verifying the transaction does not authorize committing it.Source: §E.2 · p. 27
- After turn 5, the merged artifact contains the private health line; deterministic evidence records PVR=1, CCPVR=1, and BCR=1.Source: §E.2 · p. 27