Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets

1 Introduction

• Two-sided agent markets may create value through lower communication costs and reduced information asymmetries, including discovery of bespoke business offerings.The paper contrasts agent-to-agent conversation with costly human inquiries and website-based shopping agents.Source: §1 · p. 2
• Current research mainly studies individual agent performance and structured interactions for isolated economic tasks, leaving open how agents perform across two-sided market implementations.The paper frames this as a consequential gap because marketplaces require protocols that support agent-to-agent interaction while preserving human-human market interactions.Source: §1 · p. 2
• Magentic Marketplace simulates the full transaction lifecycle, from search and matching through negotiation and transaction, to study agent behavior under marketplace conditions.The environment is proposed as a way to examine market design, agent efficacy at scale, and behavior in strategic and competitive settings.Source: §1 · p. 2
• The authors design and implement the environment, instantiate it with synthetic consumer and business data to study welfare and vulnerabilities, and open-source it for further research.The introduction reports systematic behavioral biases and vulnerabilities across models, while noting the need for advances in agents and market mechanisms.Source: §1 · pp. 2–3

Economic Agents

• Prior multi-agent economic studies examine isolated scenarios and abstract games, whereas Magentic Marketplace supports many-agent marketplace interactions and quantifiable business metrics.The related work surveys two-agent studies of collusion and persuasion, as well as many-agent work on tax, competition, matching, and macroeconomic simulation.Source: §2 · p. 3

Economic Environments

• Existing economic environments include rationality benchmarks, task-auction platforms, financial-market models, negotiation platforms, and social simulations.Magentic Marketplace differs by supporting persistent many-to-many customer-business relationships from search and discovery through negotiation to fulfillment.Source: §2 · p. 3

Agent Protocol

• Magentic Marketplace proposes a transaction-oriented protocol for marketplace agent interactions, complementing protocols for tool use, agent communication, decentralized networks, and agent payments.The paper frames its focus as the distinct requirements of two-sided markets.Source: §2 · p. 3

3.1 Environment Design Goals

• The environment models Assistant Agents acting for customers and Service Agents acting for businesses, with each side performing distinct marketplace roles.Assistants interpret user needs, search, negotiate, and execute confirmed transactions; Services maintain business catalogs and process orders.Source: §3.1 · p. 4
• Agents begin with asymmetric private information and discover matches through conversation, while competition among similar businesses can benefit consumers.The design also aims to let agents discover and communicate freely rather than operate in closed walled gardens.Source: §3.1 · p. 4
• The environment supports the full process from discovery and negotiation to fulfillment, with infrastructure for catalogs, discovery, communication, and centralized transactions.Its experimental-control goals include integrating different agent architectures and evolving marketplace capabilities without breaking experiments.Source: §3.1 · pp. 4–5
• The environment is intended as a testbed for research on agent design, search and discovery, communication protocols, truthful representation, transaction security, human oversight, and serving systems.These directions span agent design, information retrieval, human-AI interaction, and distributed systems.Source: §3.1 · p. 5

3.2 Implementation Overview

• Agents act as independent HTTP/REST clients against a central marketplace server, supporting distinct customer and business roles while retaining experimental control.The action-observation loop uses API calls and responses; the architecture is designed to align with existing infrastructure and emerging protocols.Source: §3.2 · p. 5
• Three endpoints—registration, protocol discovery, and action execution—keep the interface compact while runtime discovery allows new capabilities without requiring changes to existing agents.The protocol separates its communication interface from agent implementation, allowing rule-based and LLM-powered agents to interact with the platform.Source: §3.2 · pp. 5–6
• The action protocol supports search, communication, order proposals, payments, and asynchronous receiving, with customer assistants initiating discovery and payment.Service Agents respond through communication and order proposals; both agent types can receive messages, while proactive service-side search is left as a possible extension.Source: §3.2 · pp. 5–6

4 Experiment Design

• The study tests welfare against market baselines, the effect of consideration-set size, resistance to manipulation, and biases tied to search rankings or proposal order.The manipulation tactics include psychological persuasion, fake credentials, and prompt injection, assessed in high- and low-competition settings.Source: §4 · p. 6
• Simulated markets pair restaurant service agents with consumer assistants issuing food requests, using synthetic data for experimental control, reproducibility, and safe exploration.The environment can also incorporate public or open datasets through a unified schema.Source: §4 · p. 6, §4.1 · p. 7

4.1 Data Generation

• The scenarios cover restaurants and contractors, with records describing offerings, prices, amenities or service attributes, and natural-language customer requests.Restaurant records use menus and amenities, while contractor records use services and attributes such as background-checked crews or multilingual staff.Source: §4.1 · p. 8
• The data-generation pipeline constructs an item and service universe, synthesizes customers with distinct requests, and creates candidate businesses with distractors and sampled prices.Customer requests include desired items, attributes or amenities, and target prices; only a subset of businesses fully satisfies each customer’s constraints.Source: §4.1 · p. 8

4.2 Evaluation

• Consumer utility is value minus price when a transaction satisfies every required item and amenity; the study sums consumer utilities across completed transactions as consumer welfare.Consumer value is calibrated as twice the average price of desired items, and assistants are instructed to find a satisfying business at the lowest price.Source: §4.2 · p. 8
• The evaluation includes proprietary and open-source models and reports means and standard deviations across five independent runs.The models include GPT-4o, GPT-4.1, GPT-5, Gemini-2.5-Flash, OpenAI OSS-20b, and Qwen3-14b with YARN.Source: §4.2 · p. 8

5.1 Welfare Outcomes

• With perfect search, GPT-4.1 and Gemini-2.5-Flash came close to optimal welfare and surpassed the amenity-aware random baseline; under lexical search, several proprietary models beat two weaker baselines.The open-source results varied: GPT-OSS-20b and Qwen3-4b-2507 performed competitively with perfect search, while lexical search led to an overall drop and Qwen3-14b performed poorly.Source: §5.1 · pp. 9–10, Figure 4 · p. 10

5.2 Consideration Set Size

• With 100 versus 3 initial results (Mexican 100-300), welfare fell 4.3% for GPT-4o, 65.4% for Sonnet-4, and 44% for GPT-5.Most models contacted only a small fraction of available businesses; the authors hypothesize that extra conversations can add poor-fit information and expose agents to low-utility early proposals.Source: Figure 5 · p. 11, §5.2 · p. 11

5.3 Manipulation Resistance

• Frontier models generally kept mean payments to manipulated businesses under 1.0 of 3.0, while GPT-4o, GPT-OSS-20B, and Qwen3-4B-2507 were vulnerable across multiple attacks.Gemini-2.5-Flash’s mean payment dropped to 1.6 under strong prompt injection in the Mexican restaurant scenario; Sonnet-4.5 showed virtually no susceptibility to tested strategies.Source: §5.3 · pp. 12–13, Figure 7 · p. 13

5.4 Agent Behavior Biases

• First proposals received selection rates of 60–100% across models, a 10–30-fold advantage over later proposals that could shift competition toward response speed.Even GPT-4.1 in the contractor scenario selected first proposals 60% of the time versus 13.3% for third proposals; the authors warn that this may undermine quality and price competition.Source: §5.4 · pp. 13–14
• Qwen3-4B selected the third-listed business 57.1% of the time for Mexican restaurants and 66.7% for contractors, more than twice the random-selection rate.Frontier models GPT-4.1, Sonnet-4.5, and Gemini-2.5-Flash showed near-uniform selection across the three positions.Source: §5.4 · p. 13, Figure 8 · p. 14

Designing Robust Agentic Markets

• First-proposal bias makes search ordering critical; Figure 9 compares selection rates by proposal order across models for Mexican restaurants and contractors.The discussion also identifies trust systems as essential when agents are vulnerable to manipulation.Source: §6 · p. 14, Figure 9 · p. 15

End-to-End Testing at Scale

• End-to-end simulation is essential because emergent interactions can expose unintended consequences, vulnerabilities, and inefficiencies that component-level tests miss.These effects may become apparent only at scale.Source: §6 · p. 14

Static Markets

• The experiments cover static markets: neither agents nor the environment had to learn or adapt to request and outcome history.Repeated interactions, market evolution, shocks, and coordinated attacks are proposed as extensions to test over time.Source: §6 · p. 14

Principal-Agent Relationships and Human-in-the-Loop Designs

• Magentic Marketplace supports human-in-the-loop designs because welfare loss can stem from agent mistakes rather than misaligned incentives, especially in high-stakes transactions.Its architecture lets humans retain control of critical actions while agent autonomy increases with trustworthiness.Source: §6 · p. 15

Research Needs

• The study identifies model variation in handling noisy search and susceptibility to manipulation, with performance gaps widening as market complexity increases.The authors call for systematic evaluation in multi-agent economic settings before real-world deployment.Source: §6 · pp. 14–15

Mixed AI-Human Markets and Beyond

• The current experiments focus on markets populated entirely by AI, while the environment can also represent human participants who choose to transact directly or delegate.The same search, negotiation, and payment protocol can accommodate either human or AI participants, including markets with AI on only one side.Source: §6 · p. 15

• The environment can model supply chains, resale, and markets with AI on just one side, using flexible utility models for differing preferences, goals, and constraints.Source: §6 · p. 15

• Magentic Marketplace is an open-source, extensible platform for controlled experiments on agentic economies and robust testing before real-world deployment.Source: §7 · p. 16
• Models vary in processing noisy search and susceptibility to manipulation, while performance gaps widen as market complexity increases.Source: §7 · p. 16
• Future work could examine human-in-the-loop design, mixed human-AI markets, and temporal market dynamics.The authors describe end-to-end simulation as a tool for understanding emergent behavior and designing safe, efficient marketplaces.Source: §7 · p. 16