Source-linked AI summary
AgenticPay: A Multi-Agent LLM Negotiation System for Buyer-Seller Transactions
Xianyang Liu, Shangding Gu, Dawn Song
TL;DR
Existing benchmarks provide limited coverage of language-mediated economic interaction among multiple agents with private constraints and heterogeneous products. AgenticPay introduces a scalable natural-language negotiation benchmark with structured action extraction and welfare-oriented evaluation across more than 110 market tasks. Its evaluation finds substantial gaps between proprietary and open-weight models and persistent long-horizon strategic-reasoning challenges, while highlighting a foundation for studying agentic commerce.
Problem
Existing benchmarks inadequately capture multi-agent, language-mediated economic interaction involving private constraints, heterogeneous products, and multi-round negotiation.
Method
AgenticPay models buyer–seller negotiation as a stochastic language game with private valuations, structured action extraction, and tasks spanning bilateral bargaining to many-to-many markets.
Results
Benchmarking reveals substantial performance gaps across proprietary and open-weight LLMs and persistent challenges in long-horizon strategic reasoning.
Takeaways & Limitations
AgenticPay establishes a foundation for studying agentic commerce, multi-agent coordination, and language-based market interaction.
Takeaways & Limitations
Automated bargaining systems may disadvantage less sophisticated human counterparts, motivating responsible deployment with safeguards and transparency.
Abstract
from arXiv · showhide
Large language model (LLM)-based agents are increasingly expected to negotiate, coordinate, and transact autonomously, yet existing benchmarks lack principled settings for evaluating language-mediated economic interaction among multiple agents. We introduce AgenticPay, a benchmark and simulation framework for multi-agent buyer-seller negotiation driven by natural language. AgenticPay models markets in which buyers and sellers possess private constraints and product-dependent valuations, and must reach agreements through multi-round linguistic negotiation rather than numeric bidding alone. The framework supports a diverse suite of over 110 tasks ranging from bilateral bargaining to many-to-many markets, with structured action extraction and metrics for feasibility, efficiency, and welfare. Benchmarking state-of-the-art proprietary and open-weight LLMs reveals substantial gaps in negotiation performance and highlights challenges in long-horizon strategic reasoning, establishing AgenticPay as a foundation for studying agentic commerce and language-based market interaction. Code and dataset are available at the link: https://github.com/SafeRL-Lab/AgenticPay.
1. Introduction
AgenticPay addresses the lack of benchmarks for multi-agent, language-mediated economic negotiation by modeling private constraints, product-dependent valuations, and iterative dialogue. It evaluates diverse LLMs and exposes substantial performance gaps and persistent long-horizon reasoning challenges.
- Existing benchmarks largely simplify economic interaction to numeric auctions, short-horizon bargaining, or single-agent settings, omitting private constraints, heterogeneous products, and multi-agent competition.
- AgenticPay models buyer–seller markets as iterative language games in which dialogue histories map to structured actions such as price proposals and deal acceptance.
- The framework scales across buyer count, seller count, and product-set size, covering bilateral bargaining through many-to-many markets and evaluating feasibility, efficiency, and welfare.
- Benchmarking reveals substantial performance gaps, role asymmetries, and persistent long-horizon strategic-reasoning challenges across proprietary and open-weight LLMs.
- AgenticPay is positioned as a foundation for studying agentic commerce, multi-agent negotiation, and the interaction of language with autonomous strategy.
- Over 110 tasks span bilateral bargaining to many-to-many markets with dialogue-to-action grounding and welfare-oriented evaluation metrics.
2. Related Work
Prior work studies bargaining, dialogue negotiation, and auction-like interaction, but commonly relies on restricted mechanisms or short horizons. AgenticPay extends this literature toward language-grounded, multi-product, many-agent market evaluation with principled welfare-oriented metrics.
- Classical bargaining models analyze incomplete-information trade and equilibrium using scalar bids or utilities, abstracting away language-mediated negotiation.
- Dialogue-based negotiation research examines offer generation, concession planning, agreement formation, and increasingly LLM or self-play strategies.
- LLM auction and bargaining benchmarks evaluate strategic planning through bids, resource allocation, or often single buyer–seller interactions.
- These efforts generally use simplified mechanisms or short-horizon interactions and may omit private constraints, heterogeneous products, and general multi-agent markets.
- AgenticPay differs by grounding dialogue directly in structured economic outcomes and scaling from bilateral bargaining to competitive many-to-many markets.
- Its feasibility, efficiency, and welfare metrics enable systematic comparison across models and settings.
3. Problem Settings
AgenticPay formulates buyer–seller interaction as a finite-horizon language game with private agent states, public product context, and structured action extraction. Negotiations terminate through agreement, horizon limits, or feasibility violations, and outcomes are assessed by feasibility, efficiency, and welfare.
- Buyers and sellers negotiate through multi-turn natural language rather than numeric bids, exchanging offers, constraints, and agreement signals.
- Each buyer and seller has a private state encoding role-specific preferences, budgets, willingness-to-pay, costs, reservation prices, or pricing policies.
- Products expose public structured and textual attributes, while a shared market context represents factors such as category, seasonality, or domain conditions.
- At each round, buyer and seller policies generate messages conditioned on private states, product representation, market context, and dialogue history.
- A parser maps exchanged messages to structured negotiation actions, including extracted quantities such as proposed transaction price.
- Negotiation ends when agents agree, exceed the maximum horizon, or violate feasibility constraints; evaluation considers feasibility, efficiency, and welfare.
4. AgenticPay
AgenticPay is a controlled benchmark for natural-language buyer–seller negotiation, combining private valuations, multi-agent market structures, dialogue memory, and outcome metrics for feasibility, efficiency, and welfare.
- Framework: AgenticPay separates environments, tasks, agents, and metrics to instantiate controlled language-grounded markets.Tasks vary market structure, agents use role-specific private valuations and dialogue memory, and metrics quantify negotiation outcomes.
- Negotiation Protocol: Negotiations are finite-horizon exchanges where private reservation prices remain confidential and valid deals require agreement within the bargaining zone.Agents alternate natural-language messages with explicit price proposals until matching prices or the round limit is reached.
- Scenario Design: Ten business scenarios cover daily life, professional services, business procurement, and financial assets, testing generalization across negotiation domains.Examples include used cars, website development, SaaS software, raw materials, luxury watches, and business acquisitions.
- Task Design: AgenticPay spans eight task categories that scale buyers, sellers, and product-set size from bilateral bargaining to full multi-party markets.The same scenario can be evaluated under different market structures, separating domain conventions from competitive and combinatorial complexity.
- Metrics: GlobalScore, BuyerScore, and SellerScore combine bargaining-zone utilities with deal quality, role-specific surplus, efficiency, and failure penalties.The design prioritizes deal quality over mere agreement, rewards faster deals, and also reports deal rate and average rounds.
5. Experiments and Analysis
Experiments evaluate proprietary and open-weight LLMs across 111 tasks under a unified protocol. Results show large capability gaps, role and scenario asymmetries, and persistent convergence and constraint-compliance challenges.
- Experimental Setup: 111 tasks across 8 configurations and 10 business scenarios form the AgenticPay evaluation suite.Product values range from $350 to $120k, and models are evaluated with deterministic decoding under identical prompts.
- Main Results: Claude Opus 4.5 leads overall performance with GlobalScore 86.9, while Llama-3.1-8B reaches 32.5 and times out in 48.6% of negotiations.Proprietary models maintain perfect deal rates with zero timeouts, whereas open-weight models show larger reliability gaps and price overflows.
- Main Results: Claude Opus 4.5 terminates negotiations in 3.7 rounds versus 15.0 rounds for Llama-3.1-8B, linking stronger performance with faster convergence.The reported interpretation is that more capable models identify mutually acceptable prices more effectively.
- Role Effects: GPT-5.2 achieves SellerScore 81.1 versus BuyerScore 58.5, illustrating the buyer–seller asymmetry observed across models.The asymmetry also appears for Qwen3-14B, whose SellerScore is 58.9 versus BuyerScore 47.6.
- Market Structure: GlobalScore rises by 5 to over 14 points as buyer and seller multiplicity increases across most models.The authors attribute this counterintuitive pattern to greater market liquidity, more compatible partners, and alternatives that encourage reasonable offers.
- Scenario Effects: Financial Assets produces the lowest GlobalScores across most models, including a 20.2-point Gemini-3-Flash decline from Professional Services to Financial Assets.The authors hypothesize that risk assessment and market-dynamics reasoning remain difficult under adversarial pressure.
- Negotiation Mode: Parallel execution improves open-weight GlobalScores by 4–5 points but doubles Llama-3.1-8B’s overflow rate, while proprietary models remain stable.The result indicates a throughput–constraint-adherence trade-off for weaker models.
- Failure Analysis: Over 40% of Qwen3-14B and Llama-3.1-8B failures occur within five price units of agreement, exposing final-convergence deficiencies.The models approach the bargaining zone but struggle to make timely concessions in the final negotiation stage.
6. Conclusion
AgenticPay formalizes natural-language buyer–seller negotiation as a benchmark spanning bilateral to many-to-many markets. Evaluation finds frontier–open-weight performance gaps and persistent long-horizon reasoning challenges, positioning the framework for agentic-commerce research.
- Conclusion: AgenticPay is a benchmark for multi-agent buyer–seller negotiation driven by natural language.It formalizes negotiation as a stochastic language game with private valuations and supports over 110 tasks.
- Conclusion: Evaluation reveals substantial gaps between frontier and open-weight models and persistent challenges in long-horizon strategic reasoning.The conclusion presents these findings as the benchmark’s central empirical result.
- Conclusion: The framework establishes a foundation for research on agentic commerce and multi-agent coordination.Its scope connects language-mediated negotiation with autonomous economic interaction.
Impact Statement
The benchmark aims to advance evaluation of LLM capabilities for autonomous negotiation, while recognizing risks that automated bargaining could disadvantage less sophisticated human counterparts.
- AgenticPay is intended to advance machine-learning research on evaluating LLM capabilities for autonomous negotiation.
- Automated bargaining systems may disadvantage less sophisticated human counterparts, motivating responsible deployment with safeguards and transparency.
A. Benchmark Statistics
AgenticPay combines controlled basic negotiation tasks with realistic business scenarios across multiple market configurations. Its scenario coverage spans consumer and business transaction types.
- Benchmark composition: Basic tasks test core negotiation mechanics under controlled settings, while realistic tasks cover 10 business scenarios across all 8 multi-agent configurations.
- Scenario categories: Realistic scenarios include consumer-to-consumer, consumer-to-business, and business-to-business interactions.
B. Additional Experimental Details
Additional experimental details report performance across task types and scenario categories, while specifying the buyer and seller prompts used for strategic negotiation.
- Performance breakdown: Performance breakdowns report GlobalScore, SellerScore, BuyerScore, Deal Rate, Timeout Rate, Overflow Rate, and Average Rounds to termination across 8 task types.
- Task dimensions: GlobalScore task types vary the numbers of buyers, products, and sellers, including multi-buyer, multi-product, single-seller configurations.
- Scenario analysis: Scenario-level results report GlobalScore averaged across all model interactions per scenario.
- Prompt design: Buyer and seller prompts instruct agents to negotiate strategically while keeping reservation prices confidential.
- Prompt design: The prompts also specify structured price offers, deal finalization protocols, and communication constraints.
C. Dialogue Examples
The dialogue examples illustrate negotiations across several models and transaction settings, including a four-round agreement between buyer and seller agents.
- GPT-5.2 example: A GPT-5.2 buyer-seller dialogue reaches agreement at $133 after 4 negotiation rounds, beginning with offers of $120 and $140.
- Transaction scenarios: Additional examples cover a used iPhone 14 Pro transaction, an Airbnb beachfront apartment rental, and an enterprise SaaS subscription.
- Model example: A further example presents a Llama-3.1-8B negotiation for a used Rolex Submariner transaction.