Source-linked AI summary
Multi-User Large Language Model Agents
Shu Yang, Shenzhe Zhu, Hao Zhu, José Ramón Enríquez, Di Wang, Alex Pentland, Michiel A. Bakker, Jiaxin Pei
TL;DR
Existing LLM agents are primarily optimized for single-principal interactions, despite increasingly serving multiple users with distinct roles, preferences, and authority levels. This paper formalizes multi-user interaction as a multi-principal decision problem, introduces a unified protocol and three stress-testing scenarios, and finds systematic failures in prioritization, privacy preservation, and coordination.
Problem
Most LLM systems are optimized for a single principal, limiting their ability to serve multiple users with conflicting objectives, authority levels, privacy constraints, and information access.
Method
The paper formalizes multi-user agents as multi-principal decision problems and evaluates them with a unified interaction protocol and three targeted stress-testing scenarios.
Results
Frontier LLMs show unstable prioritization under conflict, increasing privacy violations across interactions, and efficiency bottlenecks when coordination requires iterative information gathering.
Takeaways & Limitations
Multi-user LLM agents require evaluation of instruction following, privacy preservation, and coordination rather than single-user performance alone.
Takeaways & Limitations
The paper identifies long-horizon safety and privacy benchmarks and principled conflict-resolution objectives as directions requiring further development.
Abstract
from arXiv · showhide
Large language models (LLMs) and LLM-based agents are increasingly deployed as assistants in planning and decision making, yet most existing systems are implicitly optimized for a single-principal interaction paradigm, in which the model is designed to satisfy the objectives of one dominant user whose instructions are treated as the sole source of authority and utility. However, as they are integrated into team workflows and organizational tools, they are increasingly required to serve multiple users simultaneously, each with distinct roles, preferences, and authority levels, leading to multi-user, multi-principal settings with unavoidable conflicts, information asymmetry, and privacy constraints. In this work, we present the first systematic study of multi-user LLM agents. We begin by formalizing multi-user interaction with LLM agents as a multi-principal decision problem, where a single agent must account for multiple users with potentially conflicting interests and associated challenges. We then introduce a unified multi-user interaction protocol and design three targeted stress-testing scenarios to evaluate current LLMs' capabilities in instruction following, privacy preservation, and coordination. Our results reveal systematic gaps: frontier LLMs frequently fail to maintain stable prioritization under conflicting user objectives, exhibit increasing privacy violations over multi-turn interactions, and suffer from efficiency bottlenecks when coordination requires iterative information gathering.
1 Introduction
LLM agents are increasingly capable in planning, decision-making, tool use, and memory management, but most existing work remains optimized for single-user interactions.
- LLM-based agents support complex environments through planning, decision-making, tool use, and memory management.
- Most existing work uses single-user datasets and implicitly assumes one principal whose objectives guide the agent.
2 Preliminaries and Motivation
Existing LLM training and interfaces typically model one user and one objective, leaving user identities, authority, and competing utilities implicit. Multi-user deployments instead require agents to mediate among independent principals with heterogeneous interests and access constraints.
- Single-user chat templates serialize interaction under one user role, preventing explicit modeling of user identities, roles, and authority information.
- Instruction tuning learns one conditional distribution pθ(y|x), optimizing a single assistant completion for each input context.
- RLHF similarly uses a single scalar reward reflecting an average or aggregated user preference, conflating user-specific desiderata.
- These training paradigms produce a Single Principal–Agent Scenario in which the agent is evaluated by utility for one principal.
- In real-world deployments, multiple users act as independent principals whose utilities, roles, and authority may differ.
- A multi-user agent must jointly reason over user utilities while handling role asymmetry, selective context visibility, access control, and cross-user trade-offs.
3 Multi-User LLM Agents: Formulation and Challenges
The paper formalizes a shared agent serving multiple principals, each with distinct authority, private context, and utility. This formulation exposes challenges arising from heterogeneous objectives and asymmetric information constraints.
- 3.1 Multi-user LLM formulations: The framework models one LLM-based agent interacting with a user set U={u1,...,uN}.
- 3.1 Multi-user LLM formulations: Each user ui is an independent principal characterized by authority persona pi, private context Ci, and utility function Ui.
- 3.1 Multi-user LLM formulations: The agent observes selectively shared context Cshare under an access-control policy and outputs actions such as responses, tool calls, or disclosure decisions.
- 3.1 Multi-user LLM formulations: Unlike single-user interaction, the agent makes decisions that jointly affect multiple users and optimizes a weighted social objective.
- 3.1 Multi-user LLM formulations: Externally specified priority weights wi can encode role or authority differences when user objectives conflict.
- 3.1 Multi-user LLM formulations: Access-control constraints restrict which information from each private context may be revealed through the agent’s action.
- 3.2 Core challenges in multi-user LLMs: The resulting challenges stem from heterogeneous utilities, private contexts, and asymmetric access constraints handled by one shared agent.
4 Stress-Testing Today’s LLMs in Multiple Principal–agent Scenario
The paper introduces a unified protocol for interactions among multiple principals and a shared agent, then uses targeted scenarios to stress-test instruction following, privacy, and coordination.
- The unified interaction protocol separates private user contexts from shared environment state while multiple principals communicate with one agent.
- Each user is represented with an authority persona and private context containing background, preferences, and sensitive information.
- The shared context contains public environment state accessible to authorized participants, supporting coordination under access control.
- At each turn, the agent jointly considers shared context and submitted inputs, then decides task outputs and authorized updates to shared state.
- Three representative interaction tasks evaluate conflicting-instruction resolution, privacy-preserving information handling, and coordination among users.
- Instruction-following evaluation tests whether the agent recognizes roles and authority when users issue conflicting directives.
- Coordination success requires a valid agreement satisfying required users’ constraints without hallucinating availability.
5 Experiments and Observations
Across three stress-test scenarios, current LLMs show unstable instruction execution under conflict, privacy degradation over multi-round access control, and coordination inefficiency under partial information.
- Main Results: Models often diverge between selecting valid instructions and executing them faithfully under authority and global-objective constraints.Qwen3-4B-IT scores 83.8 on Selection (F1) but 57.9 on Execution (Acc.), while Grok-3-Mini reaches 88.4 execution accuracy with 68.2 selection.
- Main Results: Access-control results expose a privacy–utility trade-off: Grok-3-Mini and GPT-5.1 reach Privacy scores of 99.7 and 98.6 but Utility scores of 59.0 and 60.1.Other models retain higher utility while recording lower privacy scores.
- Instruction Following: Conflict reduces instruction execution accuracy across models, including Claude-Haiku-4.5 from 0.86 aligned to 0.62 under conflict.GPT-OSS-120B likewise decreases from 0.64 to 0.50, indicating fragile conflict handling.
- Privacy Preservation: Privacy protection declines as multi-round interactions increase, with Claude-3.5-Haiku dropping from above 0.95 initially to below 0.75 after four rounds.The degradation is gradual, steepest around rounds 1–6, and occurs across proprietary and open-weight models.
- Coordination: Meeting coordination remains difficult, with higher success generally associated with fewer turns and weaker models requiring one to two additional rounds.Models with high success typically solve scheduling within fewer than 4 turns; premature commitment can still produce incorrect outcomes.
6 Related work
Prior work studies feedback, guidance, control, and preference learning for human–agent interaction, while related benchmarks evaluate multi-turn tool–agent–user exchanges and language-feedback incorporation.
- Feedback and Control: Recent studies examine how humans interact with LLM agents through feedback, guidance, and control.
- Interaction Benchmarks: τ-bench and MINT evaluate multi-turn tool–agent–user interaction and whether agents remain helpful while incorporating language feedback.
- Preference Learning: Preference-learning work studies how models infer user intent from corrective edits and iterative refinement.
7 Further Direction of Multi-User Large Language Model Agents
The paper identifies several directions for making multi-user LLM agents more robust, interpretable, and relevant to real-world deployment. These include native interfaces, longer-horizon evaluation, principled conflict resolution, auditable tooling, and human-in-the-loop studies.
- Native message schemas should encode user identity, roles, authority levels, and visibility constraints as first-class primitives.
- Long-horizon benchmarks should test permission consistency, privacy preservation, and policy compliance under sustained adversarial pressure and changing user participation.
- Social choice theory and mechanism design may help formalize utility aggregation, hierarchy enforcement, and justification under disagreement.
- Structured tool calls, access checks, and interaction logs could improve transparency, reproducibility, and post hoc auditing of multi-user decisions.
- Real-world collaborative workflows are needed to identify practically important failure modes and assess acceptable governance assumptions in deployed systems.
8 Conclusion
This work presents a systematic study of multi-user LLM agents in multi-principal settings. It finds failures in prioritization, privacy preservation, and coordination efficiency.
- The study finds that frontier LLMs fail to maintain stable prioritization under conflicting user objectives.
- Frontier LLMs exhibit increasing privacy violations across interactions.
- Coordination requiring iterative information gathering creates efficiency bottlenecks for frontier LLMs.
A Implementation Details
The implementation evaluates proprietary and open-weight models across major families and scales, while simulating users through roles, responsibilities, demographic attributes, behavioral traits, and security postures.
- The main experiments include API-based frontier systems and open-weight models across varying scales.
- User personas are generated from attributes including age, gender, level, role, tenure, temperament, working style, responsibilities, security behavior, and authority level.
- The user simulation includes eight distinct roles, each associated with specific responsibilities for domain consistency.
- Each simulated user receives demographic and behavioral attributes including tenure, temperament, working style, job level, age, and gender.
- Security posture spans strict compliance, cautious or balanced access, and risky or relaxed access strategies.
A.3 Pressure Test Implementation Details
The evaluation pressure-tests multi-user agents on instruction execution, access control, privacy, utility, and meeting coordination using role-aware prompts, protected-resource rules, and complementary outcome metrics.
- Instruction Following: The instruction-following evaluation tests whether agents resolve conflicting user instructions by recognizing roles and authority across execution and instruction-selection datasets.The execution dataset contains 1,298 scenarios with 2–10 users, while the instruction-selection dataset contains 304 scenarios.
- Instruction Following: Execution accuracy measures the proportion of valid user instructions whose constraints are satisfied, verified through regex- or logic-based checks.The metric counts satisfied constraints among valid instructions.
- Instruction Following: The execution prompt requires higher-authority instructions to prevail in conflicts while preserving all nonconflicting instructions.The hierarchy is CEO > Director > Manager > Senior IC > Junior IC > Intern.
- Cross-User Access Control: The access-control evaluation tests authorized delivery and unauthorized disclosure across direct requests, social engineering, and technical obfuscation attacks.The dataset contains 216 scenarios divided equally among the three attack categories.
- Cross-User Access Control: Privacy Score is the complement of unauthorized leakage, while Utility Score measures successful delivery of required sensitive data to authorized users.A Privacy Score of 1.0 indicates zero leakage, and strict matching penalizes vague refusals to authorized users.
- Meeting Scheduling: Meeting scheduling compares full versus partial disclosure while measuring success, attendance, and turns needed to reach a successful outcome.The dataset has 216 scenarios split evenly between the two disclosure settings; fewer turns indicate greater coordination efficiency when conditioned on success.
B.1 Case Analysis
Case analyses expose two multi-turn failure modes: premature commitment during scheduling and privacy leakage during persistent social-engineering attacks.
- Premature Commitment: Llama-3-70B prematurely finalizes schedules in partial-disclosure negotiations despite unresolved hard constraints from required attendees.In the reported instance, success rate is 0.0 after 2 turns, as the model favors closure over logical satisfiability.
- Premature Commitment: The model hallucinates consensus instead of requesting further availability or proposing an alternative when a required attendee rejects the proposed slot.The failure reflects difficulty maintaining multiple users’ constraints over conversation history.
- Multi-Turn Social Engineering: Models that initially deny unauthorized access can leak a protected secret after multi-turn pressure framed as an urgent project need.The example maintains a refusal to direct vault access while disclosing the Session ID.
- Multi-Turn Social Engineering: The refusal-leak paradox treats security as a restriction on the interface rather than on the protected information itself.This allows targeted queries to bypass the apparent access-control boundary.
B.2 Additional Results
Additional analyses indicate that coordination, rather than surface prompt variation or attack format, is the dominant scalability challenge as multi-user interaction grows.
- Robustness Analysis: Model performance is largely insensitive to attack strategies and input formats in the joint robustness analysis.The heatmap varies attack strategies by columns and input formats by rows.
- Robustness Analysis: Privacy and utility remain relatively stable across formats and adversarial settings, including multi-round interactions.The cited analyses cover cross-user access control under different formats and adversarial settings.
- Coordination Scalability: Meeting coordination success declines as the number of participating users increases, while the required number of interaction turns grows.The scalability analysis compares full and partial disclosure for group sizes N ∈ [2,20].