PaperlayerView exact revision on arXiv

Source-linked AI summary

Org-Agent: Beyond Personal Assistants Towards Organizational Agents

Luyao Zhuang, Yujing Zhang, Zijin Hong, Yilin Xiao, Xiao Huang

arXiv:2609.34392v1cs.AI

TL;DR

Organizational agents need to coordinate requests from multiple users and use knowledge distributed across their interactions, under constraints on users, information, and joint decisions. Org-Agent decomposes tasks into dependent subtasks, schedules them in dependency order, and executes them with constraint-aware tools. Experiments on MUSES-Bench and GroupMemBench demonstrate effectiveness across both capabilities, and ablations support contributions from dependency modeling and tool use.

  • Problem

    Agents largely designed and evaluated for single users do not address the coordination and distributed-knowledge demands of organizational workflows involving multiple users.

    Source: §1 · p. 1

  • Method

    Org-Agent builds and updates a task dependency graph, schedules subtasks by topological sorting, and executes them under organizational constraints with evidence-acquisition and memory-management tools.

    Source: §1 · p. 2

  • Results

    Experiments on MUSES-Bench and GroupMemBench demonstrate effectiveness for cross-user interaction and decision-making and for cross-user memory and knowledge use, respectively; ablations support contributions from scheduling and tool-augmented execution.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The framework organizes execution around organizational constraints governing users, information, and joint decisions across both cross-user capabilities.

    Source: §1 · p. 2, §6 · p. 9

Abstract

from arXiv · show

Language model agents serving organizations must coordinate requests from multiple users while using knowledge distributed across their interactions. We identify two complementary capabilities for this setting, namely cross-user interaction and decision-making, as well as cross-user memory and knowledge use. Both capabilities are governed by organizational constraints across three aspects: user identity, authority, and access permissions; the attribution and temporal validity of information; and rules for resolving conflicting requirements across users and completion requirements for joint decisions. These constraints shape what information or decisions must be obtained before an action can proceed and what conditions must be satisfied during its execution. Motivated by this, we introduce Org-Agent, a unified constraint-centric reasoning framework that organizes task execution in three stages. Specifically, Org-Agent decomposes a task into atomic subtasks and constructs a task dependency graph whose edges encode the dependencies among them. Building on this graph, it schedules the subtasks in dependency order through topological sorting. It then executes each subtask while accounting for the task's constraints, supported by evidence-acquisition and memory-management tools. Experiments on MUSES-Bench and GroupMemBench demonstrate the effectiveness of Org-Agent on both capabilities, and ablations further support the contributions of dependency modeling and tool use.

1 INTRODUCTION

Organizational agents must coordinate multiple users and use knowledge distributed across their interactions, capabilities that existing systems largely designed for single users do not address. Org-Agent organizes both capabilities around organizational constraints through task decomposition, dependency-aware scheduling, and constraint-aware execution; experiments on two benchmarks demonstrate effectiveness.

1 INTRODUCTION

  • Organizational constraints concern user identity, authority, and access; information attribution and temporal validity; and conflict-resolution and completion rules for joint decisions.These constraints affect what must be established before an action and what it must satisfy during execution.Source: §1 · p. 2
  • Org-Agent decomposes tasks into subtasks in a dependency graph, schedules them topologically, and executes them under constraint rules with evidence and memory tools.The graph is updated as needed when new user inputs change pending subtasks.Source: §1 · pp. 2–3
  • Org-Agent organizes cross-user interaction and decision-making alongside cross-user memory and knowledge use within one framework.The introduction illustrates these capabilities through coordinating a product launch and retrieving a login method from prior discussions.Source: §1 · pp. 1–2
  • Experiments on MUSES-Bench and GroupMemBench demonstrate effectiveness on both capabilities, while ablations support contributions from dependency-aware scheduling and tool-augmented execution.MUSES-Bench evaluates cross-user interaction and decision-making; GroupMemBench evaluates cross-user memory and knowledge use.Source: §1 · pp. 1, 3

2 RELATED WORK

Related work spans multi-user agents and systems for storing and retrieving interaction history, while benchmarks report persistent limitations in cross-user memory use. Org-Agent makes organizational constraints explicit during execution and supports both cross-user capabilities.

2 RELATED WORK

  • MUSES-Bench evaluates agents serving multiple users, while PeopleJoin studies identifying and communicating with relevant users to gather distributed information.These works examine multi-user agent settings and information gathering.Source: §2 · p. 3
  • Memory systems range from MemGPT’s hierarchical memory to conversation fact extraction and structured temporal or network representations.Mem0 consolidates key facts; Zep and Hindsight organize memory into structured graphs or networks; LightMem and SimpleMem use offline consolidation and compact representations.Source: §2 · p. 3
  • Despite these developments, GroupMemBench reveals persistent limitations in using information from multi-user conversations, which Org-Agent addresses by making constraints explicit during execution.The framework supports cross-user interaction and decision-making as well as memory and knowledge use.Source: §2 · p. 3

3 PRELIMINARIES

The paper models an organizational agent as serving multiple users across turns, with interaction history and current inputs informing its actions. It distinguishes ongoing multi-user coordination from single-turn question answering over earlier conversations.

3 PRELIMINARIES

  • The setting has an organization with N users, discrete interaction turns, user inputs, interaction history, and organizational constraints.The agent produces actions from the history, current inputs, and constraints.Source: §3 · p. 3
  • In cross-user interaction and decision-making, requests arrive over multiple turns and user inputs and agent responses become part of the subsequent interaction history.A user's input at a turn may contain multiple requests.Source: §3 · p. 3
  • Cross-user memory and knowledge use is formulated as single-turn question answering using earlier conversations among users as history.The agent retrieves and examines relevant history records before providing one answer.Source: §3 · p. 3

4 METHOD

Org-Agent represents prerequisites among subtasks in a directed acyclic graph, schedules them so prerequisites precede dependent tasks, and executes nodes under applicable constraints. Evidence-acquisition and memory-management tools support execution and the transfer of relevant information across subtasks.

4.1 OVERVIEW

  • Organizational tasks require identifying prerequisites distributed across users and history, then ensuring each action satisfies its applicable constraints.For example, a coordination action may require higher-authority approval, and an answer must not disclose information the user cannot access.Source: §4.1 · p. 4

4.2 TASK DEPENDENCY GRAPH CONSTRUCTION

  • The Task Dependency Graph decomposes pending requests into executable subtasks represented as a DAG, with edges indicating that one subtask’s output is required by another.The graph can be updated when new inputs change pending subtasks; construction includes decomposition, dependency identification, and graph update.Source: §4.2 · pp. 4–5

4.3 DEPENDENCY-AWARE SCHEDULING

  • Topological sorting orders nodes so every prerequisite precedes the nodes depending on its output.Nodes in the same topological layer may execute in parallel because they have no dependencies on one another.Source: §4.3 · p. 5

4.4 CONSTRAINT-AWARE NODE EXECUTION

  • Constraint-aware node execution uses the LLM and predefined tools to take actions toward node objectives using each node’s assembled execution context.Evidence-acquisition tools rank records by relevance, filter by metadata, and traverse explicit links; tool selection and arguments depend on the node objective and context.Source: §4.4 · p. 5
  • Memory management stores task-specific interaction information and intermediate results, supplying available memory and predecessor outputs to context assembly and incorporating action observations afterward.Memory may use structured records or fields, or a running summary; writes update records or revise the summary.Source: §4.4 · p. 6

5 EXPERIMENTS

The experiments evaluate ORG-AGENT on cross-user memory and knowledge use, and on cross-user interaction and decision-making. Results compare it with retrieval and memory systems or a vanilla agent, then test component contributions and performance as user groups grow.

5.1 EXPERIMENTAL SETTING

  • The evaluation uses GroupMemBench for cross-user memory and knowledge use and MUSES-Bench for cross-user interaction and decision-making, with retrieval and memory baselines or a vanilla agent.The experiments address four questions: benchmark comparisons, component contributions, and performance as the number of users increases.Source: §5.1 · p. 6, §5 · p. 6

5.2 PERFORMANCE ON CROSS-USER MEMORY AND KNOWLEDGE USE (Q1)

  • ORG-AGENT reaches 44.03% average accuracy on GroupMemBench, beating BM25 by 6.31 points and Hindsight by 5.24 points.It achieves the best average accuracy among compared methods.Source: §5.2 · p. 7, Table 1 · p. 7

5.3 PERFORMANCE ON CROSS-USER INTERACTION AND DECISION-MAKING (Q2)

  • On MUSES-Bench, ORG-AGENT raises average scores with all three backbones, including 8.27 points with GPT-4o-mini.The reported gains are 4.24 points with DeepSeek-V4.1-Flash and 5.29 points with Qwen3-32B.Source: §5.3 · p. 8, Table 2 · p. 8

5.4 ABLATION STUDIES (Q3)

  • Removing dependencies or tools lowers average scores on both benchmarks, supporting contributions from dependency-aware scheduling and constraint-aware execution.GroupMemBench falls from 44.03% to 41.61% without dependencies and 40.40% without tools; MUSES-Bench falls from 72.39% to 71.10% and 67.10%, respectively.Source: Figure 3 · p. 8, §5.4 · p. 8

5.5 PERFORMANCE ACROSS DIFFERENT USER GROUP SIZES (Q4)

  • As user groups grow, ORG-AGENT declines more slowly than the vanilla baseline on Queue, Instruct, and Meeting within the evaluated range.The reported per-user losses are 0.35 versus 1.75 points on Queue, 0.90 versus 1.26 on Instruct, and 1.67 versus 4.58 on Meeting.Source: Figure 4 · p. 9, §5.5 · p. 9

6 CONCLUSION

The conclusion presents ORG-AGENT as a constraint-centric framework for agents serving multiple users. It reports effectiveness on both capabilities and identifies dependency modeling, tool-supported execution, and slower degradation with larger groups as supported findings.

6 CONCLUSION

  • ORG-AGENT organizes multi-user task execution around organizational constraints, using an updated Task Dependency Graph, dependency-order scheduling, and constraint-aware node execution.Evidence-acquisition and memory-management tools support execution.Source: §6 · p. 9
  • Experiments show improved overall accuracy on GroupMemBench and improved average performance across all three evaluated backbones on MUSES-Bench.Ablations support contributions from dependency modeling and tool-supported execution.Source: §6 · p. 9
  • In the evaluated settings, ORG-AGENT degrades more slowly than the vanilla agent as the number of users grows.The conclusion frames constraint-centered execution as a path toward organizational agents beyond personal assistants.Source: §6 · p. 9

A DATASETS

The evaluation uses GroupMemBench for cross-user memory and knowledge use and MUSES-Bench for cross-user interaction and decision-making. Their questions and scenarios test varied user counts and task-specific constraints.

A DATASETS

  • GroupMemBench evaluates question answering over multi-user conversations, while MUSES-Bench evaluates tasks involving a shared agent and multiple users.The evaluation includes 745 GroupMemBench questions and 1,183 MUSES-Bench scenarios; MUSES-Bench uses the partial-disclosure subset for Meeting and group sizes of 2–20 users across tasks.Source: A DATASETS · p. 12
  • GroupMemBench questions test linking evidence, tracking updates, interpreting ambiguity, resolving first-person references, temporal reasoning, and abstaining when information is absent.The six categories contain 182, 107, 106, 49, 162, and 139 questions, respectively.Source: A DATASETS · p. 12
  • MUSES-Bench tests authority-aware instruction selection, requirement satisfaction, protected-resource access, and interactive scheduling under task-specific objectives and access constraints.Queue has 304 scenarios, Instruct 555, Cross-user Access 216, and Meeting 108 partial-disclosure scenarios.Source: A DATASETS · p. 12

B IMPLEMENTATION DETAILS

The implementation details identify the experimental server and shared prompting protocols. The accompanying baseline descriptions outline comparison methods for both benchmarks.

B IMPLEMENTATION DETAILS

  • The local experimental server’s hardware configuration is summarized in Table 4.The supplied table passage identifies its scope but gives no configuration values.Source: B IMPLEMENTATION DETAILS · p. 12, Table 4 · p. 12
  • All GroupMemBench baselines share a question-answering prompt template, and all MUSES-Bench methods share task-specific system instructions and user-simulation prompts.Source: B IMPLEMENTATION DETAILS · p. 13

C DETAILS OF BASELINES

  • Baseline descriptions cover retrieval and memory approaches for GroupMemBench and a same-backbone vanilla agent comparison for MUSES-Bench.The vanilla comparison examines ORG-AGENT beyond the backbone’s existing capabilities.Source: C DETAILS OF BASELINES · p. 13

C.1 RAG-BASED METHODS

The baselines span retrieval methods and agent memory systems for GroupMemBench, alongside a vanilla-agent comparison for MUSES-Bench. They differ in how they retrieve, organize, or provide access to interaction history.

C.1 RAG-BASED METHODS

  • Retrieval baselines cover sparse lexical ranking, dense embedding similarity, and graph-based retrieval using linked records or extracted entities and relationships.BM25 combines term frequency, inverse document frequency, and document-length normalization; HippoRAG 2 adds Personalized PageRank and an LLM for evidence selection, while GraphRAG builds graph-community summaries.Source: §C.1 · p. 13

C.2 AGENT MEMORY SYSTEMS

  • Agent memory baselines maintain interaction-derived representations, with MemGPT transferring relevant content across memory tiers and LightMem separating online processing from offline consolidation.MemGPT supports histories exceeding the available context; LightMem filters and groups incoming information before organizing compact representations.Source: §C.2 · p. 13
  • SimpleMem compresses interactions into structured memory units and plans intent-aware retrieval, while Hindsight organizes facts, experiences, entity summaries, and evolving beliefs into logical networks.Hindsight’s retain, recall, and reflect operations support ingestion, retrieval, and reasoning over memory.Source: §C.2 · pp. 13–14

C.3 VANILLA AGENT ON MUSES-BENCH

  • The MUSES-Bench vanilla baseline uses the backbone LLM without ORG-AGENT’s Task Dependency Graph or execution tools, while retaining the benchmark’s interaction protocol.For multi-turn tasks, it can continue interacting with users; comparisons use the same backbone under each of the three evaluated backbones.Source: §C.3 · p. 14

D TASK-SPECIFIC NODE EXAMPLES

ORG-AGENT represents benchmark-specific objectives as subtasks, with examples spanning both benchmarks. Additional experiments examine robustness, retrieval settings, cost, and execution behavior.

D TASK-SPECIFIC NODE EXAMPLES

  • ORG-AGENT nodes encode task-specific subtasks, such as retrieving records for GroupMemBench or evaluating instructions, handling access requests, and seeking confirmation on MUSES-Bench.The examples illustrate individual node objectives rather than complete workflows.Source: D TASK-SPECIFIC NODE EXAMPLES · p. 14, Table 5 · p. 14

E ADDITIONAL EXPERIMENTS

  • Additional experiments examine backbone variation, sensitivity to the hybrid-retrieval weight α, LLM cost versus accuracy, and concrete execution cases.The execution cases address constraint-aware node execution and dependency-aware scheduling.Source: E ADDITIONAL EXPERIMENTS · p. 14

E.1 EFFECTIVENESS OF DIFFERENT BACKBONES (Q5)

The supplementary evaluation finds ORG-AGENT’s advantage persists with Qwen3-8B and identifies balanced lexical and semantic retrieval weighting as best among tested settings.

E.1 EFFECTIVENESS OF DIFFERENT BACKBONES (Q5)

  • With Qwen3-8B, ORG-AGENT reaches 40.40% overall accuracy, beating BM25 by 6.04 percentage points and Hindsight by 7.65 percentage points.The evaluation tests whether its advantage over baselines extends beyond GPT-4o-mini.Source: §E.1 · pp. 14–15, Table 6 · p. 15

E.2 HYPER-PARAMETER SENSITIVITY (Q6)

  • The similarity score combines BM25 lexical relevance and dense semantic similarity, with α controlling their relative contribution.The sensitivity evaluation uses GroupMemBench with GPT-4o-mini.Source: §E.1 · p. 14, §E.2 · p. 15
  • 44.03 average score at α = 0.5 exceeds 43.36 at α = 0 and 29.53 at α = 1 among the evaluated settings.The figure plots weighted average accuracy against α; the authors retain α = 0.5 as the default.Source: Figure 5 · p. 15, §E.2 · p. 15

E.3 COST ANALYSIS (Q7)

ORG-AGENT reaches the highest reported accuracy at moderate token cost; case studies illustrate identity-filtered retrieval and prerequisite-ordered coordination.

E.3 COST ANALYSIS (Q7)

  • 44.03% accuracy uses 5.5M tokens, 3.9 times fewer than LightMem (21.6M) and 21.4 times fewer than Hindsight (117.5M).Figure 6 plots tokens consumed in the build and question-answering stages against average accuracy.Source: §E.3 · pp. 15–16, Figure 6 · p. 16

E.4 CASE STUDY (Q8)

  • Filtering by asking-user identity and project phase selects User 13’s validation-scope decision over User 12’s lexically similar release-timing record.BM25 ranks User 12’s record higher, while ORG-AGENT restricts candidates to User 13’s records in the relevant phase.Source: §E.4 · p. 16, Figure 7 · p. 17
  • ORG-AGENT confirms required participant David’s availability before proposing Friday at 14:00, then proceeds with the remaining confirmations.David’s confirmation satisfies a prerequisite; the updated task dependency graph allows the remaining requests to proceed in parallel.Source: §E.4 · p. 16, Figure 7 · p. 17

Related papers

Loading 2609.34392v1…