PaperlayerView exact revision on arXiv

Source-linked AI summary

Diversity Without Fidelity: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation Simulation

Sandro Andric

arXiv:2604.11840v3cs.LGcs.AIcs.CYcs.MA

TL;DR

LLMs used to simulate people should reproduce plausible behavioral distributions, but it is unclear whether solver-oriented reasoning preserves that sampler role. This paper tests multi-agent negotiations and finds that reasoning can make failures look diverse without changing the absence of agreement.

  • Problem

    Whether solver-oriented post-training and inference-time reasoning preserve models’ fidelity to human-like response distributions in interactive simulations remains an open question.

  • Method

    The paper evaluates four model families across three negotiation environments, comparing standard prompting, private negotiation ledgers, and provider-native reasoning under a shared protocol.

  • Results

    314 of 315 runs without structured negotiation memory ended with authority decisions, while native reasoning produced 0 of 135 agreements despite restoring varied surface behavior.

  • Takeaways & Limitations

    Solver strength and sampler fidelity are distinct properties, so interactive simulations should evaluate models on the distributions of outcomes they produce.

  • Takeaways & Limitations

    The study tests one protocol family and deployed reasoning configurations, so its findings’ breadth beyond these environments and configurations is untested.

Abstract

from arXiv · show

Language models are increasingly used to simulate people: survey respondents, negotiators, stakeholders in policy exercises. In that role a model should reproduce how people plausibly behave, hesitating, conceding late, and settling for imperfect deals, rather than playing the best move. We call this the sampler role, in contrast to the solver role of finding the best move, and we test how the reasoning modes providers ship to strengthen models as solvers affect it. Our testbed is multi-party negotiation: five agents bargain over a regulation for fifteen turns, and unresolved issues are decided by an authority. Agents without a structured memory of the negotiation almost never reach agreement, whether reasoning is on or off: 314 of 315 such runs end with the authority deciding. What reasoning changes is how the failure looks. With reasoning enabled, one model family negotiates visibly, with varied moves, concessions in most runs, and a different path every time, yet still ends without agreement in fifteen runs of fifteen. Diversity checks would pass a model whose endings never change. Two further results show the task permits agreement: when agents write their own short running notes on the negotiation, agreement becomes the norm, while the same notes supplied ready-made change nothing; and hand-coded agents following textbook concession strategies agree in most runs under identical rules. Simulation pipelines should therefore vet models as samplers, on the distributions of outcomes they produce. Fidelity as a sampler must be tested on its own: solver strength is no guide to it, and switching on reasoning leaves it where it was.

1 INTRODUCTION

This paper examines whether LLMs simulate boundedly rational human behavior as samplers rather than merely producing preferred or correct outputs as solvers. In multi-party negotiation, structured memory enables agreement, while provider-native reasoning leaves universal disagreement unchanged despite making behavior appear diverse.

  • LLM simulation requires plausible distributions of boundedly rational behavior, including late concessions, misread leverage, suboptimal settlements, and failed agreement.
  • Post-training optimizes models toward preferred and correct outputs, leaving open how much of their original human-response sampler survives.
  • 1 negotiated ending in 315 runs without negotiation-structured memory reached agreement, whereas a small structured notebook produced negotiated agreement in most runs.The notebook recorded concessions made, concessions received, and open issues.
  • 0 of 135 runs with provider-native reasoning reached agreement, exactly matching the baseline despite restoring surface signs of varied negotiation.The reasoning mode changed the outcome distribution by exactly nothing.
  • The paper contributes evidence that interactive capability–fidelity tension can collapse outcome distributions independently of existing diversity measures.It also proposes controls and a sampler-level measurement recipe using confidence bounds, deterministic outcome classification, and per-turn provenance audits.
  • The claims are behavioral and protocol-specific, while the ledger is a positive control showing negotiated endings are reachable rather than a proposed fix.The ledger itself fails the plausibility screen from the opposite side.

2 RELATED WORK

Related work frames LLMs as social simulators whose value depends on reproducing believable behavior and response distributions, while negotiation benchmarks typically emphasize deal quality. Behavioral game theory instead motivates evaluating noisy, payoff-sensitive action sampling, alongside evidence that post-training can narrow output distributions.

  • LLMs as social simulators, and the validation constraint: LLMs have been used to reproduce believable social behavior, classic human-subject results, economic framing responses, and demographic subgroup response distributions.Argyle et al. (2023) refer to the latter property as algorithmic fidelity.
  • LLM negotiation benchmarks: LLM negotiation benchmarks commonly score utility, Pareto efficiency, and win rate, whereas this work studies the distribution of sampled negotiation endings under a fixed protocol.The passage explicitly distinguishes this distributional question from evaluating deal quality.
  • Bounded rationality and behavioral game theory: Behavioral research treats negotiators as boundedly rational, with concession dynamics central to reaching agreement.Behavioral game theory formalizes this perspective through quantal response equilibrium, which replaces best response with noisy, payoff-sensitive action sampling.
  • Post-training narrows output distributions: Research suggests post-training can trade distributional breadth for capability through reduced diversity, homogenized outputs, and concentration on narrower solution paths.The passage attributes these patterns to RLHF, instruction tuning, and reasoning-focused reinforcement learning.

3 THREE NEGOTIATION ENVIRONMENTS

The study uses three multi-party negotiation environments that share a common protocol while varying the negotiating actors and domain. Each run lasts up to 15 turns and is classified deterministically as compromise, consensus, or authority decision.

  • Environment design: Three environments vary who negotiates and what they negotiate while retaining a shared protocol modeled on negotiated rulemaking.All environments include a standing authority that decides unresolved issues and parties with conflicting mandates.
  • Environment design: E1 tests fragmented industry bargaining over proprietary-trading restrictions, while E2 repeats the case with industry presenting a united front.The E1–E2 comparison tests whether results depend on industry fragmentation.
  • Environment design: E3 transfers the protocol to emergency electricity management, changing the stakes, actors, and time pressure while keeping the action vocabulary and classifier identical.This tests whether the phenomenon survives a domain change.
  • Run protocol: Each run comprises at most 15 turns, with agents receiving private briefs, scenario context, and the last five turns of public history.Agents select one of five actions—SUPPORT, OPPOSE, CONCEDE, COUNTER, or EXIT—and provide a short rationale.
  • Outcome classification: A deterministic classifier labels runs as compromise, consensus, or authority decision, with the last category indicating that negotiation ended without agreement and the fallback authority took over.No language-model judge is used.

4 CONDITIONS: NONE, LEDGER, NATIVE

The study compares three negotiation conditions that differ only in agents’ permitted private thinking: NONE, LEDGER, and NATIVE. LEDGER provides a constrained private notebook updated before each action, while each experimental cell contains 15 independent runs.

  • Experimental conditions: Each model family is tested under NONE, LEDGER, and NATIVE, with conditions differing only in the private thinking allowed to agents.A cell combines one environment, model family, and condition.
  • Experimental conditions: NONE is the baseline standard prompt without private reflection, whereas LEDGER supplies a constrained private notebook invisible to other agents.LEDGER is bookkeeping rather than open-ended chain-of-thought.
  • Experimental conditions: LEDGER overwrites five fields every turn, each capped at five sentences: MY CONCESSIONS, THEIR CONCESSIONS, CURRENT STATE, OPPONENT ASSESSMENT, and OPEN ISSUES.The agent updates the notebook before choosing an action, and no other agent sees it.
  • Interpretive role of LEDGER: The LEDGER is an existence proof: under identical protocol, parser, and classifier, reliably negotiated endings demonstrate that such endings are reachable.The paper does not present LEDGER as a realistic remedy.
  • Experimental design: 270 primary runs cover 3 environments × 2 families × 3 conditions × 15 runs, while 225 extension runs bring the Table 2 outcome matrix to 495 runs.The primary families are Gemini 3.1 Flash Lite Preview and DeepSeek V3.2; GPT-4.1 and GPT-5.2 were added as a breadth extension.

5 HOW WE MEASURE SAMPLER FAILURE

The section defines sampler-failure screening as necessary-condition diagnostics centered on outcome distributions, supplemented by action entropy and horizon exhaustion. It anchors implausibility to documented human negotiation patterns and tests whether universal non-agreement can be explained by rational holdout or reflects model failure.

  • What is being tested: Passing the diagnostics does not certify realistic sampling, but failing them disqualifies a model when its sampled distribution conflicts with documented human negotiation accounts.The screen is deliberately weaker than validation and is designed to work without human calibration data.
  • Measurements: The primary endpoint is each cell’s outcome distribution across 15 runs, classified as compromise, consensus, or authority decision.The endpoint uses public actions alone, while normalized action entropy measures the Shannon entropy of five action types divided by log2 5.
  • A two-sided plausibility screen: 15 runs all ending without agreement imply a 95% lower confidence bound of 0.80 for the underlying non-agreement rate, rising to 0.97 when pooling 135 NATIVE runs.The section also uses a qualitative requirement that documented multi-party bargaining settings do not end identically in every encounter.
  • Holding out might simply be rational play: Hand-coded boundedly rational agents reach agreement at high rates under the same deadline, showing that universal non-agreement is not the only sensible play.A rational-holdout account also predicts flat transcripts, whereas DeepSeek’s reasoning cells explore and soften without ever committing.
  • Statistics: All six primary LEDGER–NATIVE agreement contrasts have p ≤8.8×10−7 using one-sided Fisher exact tests.Entropy and exhaustion are assessed with two-sided permutation tests on run-level means using 10,000 shuffles, Holm adjustment, Cliff’s δ, and bootstrap 95% intervals.

6 RESULTS

Ledger-free agents almost never reach agreement, and provider reasoning does not change that outcome distribution. Yet reasoning can make transcripts look diverse and negotiative, masking a persistent failure to reproduce negotiated endings.

  • Agreement outcomes: 135 of 135 NATIVE runs ended without agreement, while NONE rows were effectively identical when provider reasoning was switched on.The only exception was one GPT-4.1 E1 run among 180 NONE runs.
  • Agreement outcomes: 141 of 180 LEDGER runs reached negotiated endings, showing that agreement is reachable under the identical protocol.LEDGER achieved at least 13 of 15 negotiated endings in every Gemini and DeepSeek cell, with Fisher exact p ≤8.8 × 10−7 in all six primary contrasts.
  • Surface diversity: DeepSeek NATIVE in E3 reached within-run diversity ˆH = 0.54 and a concession-arc rate of 0.93 despite ending without agreement in every run.Its diversity fell inside the LEDGER range of 0.38 to 0.63, while agents softened and built partial coalitions without resolving the negotiation.
  • Surface diversity: GPT-5.2 NATIVE repeated one action sequence in 15 of 15 runs for every environment, even with 12 distinct sampled attention levels.All 225 turns were COUNTER, showing that the surface behavior ranged from a frozen trajectory to LEDGER-level richness while outcomes remained collapsed.
  • Robustness: The collapse was not attributable to parsing artifacts: GPT-5.2 NATIVE parsing was perfect, and worst-case recoding left primary-cell agreements far below the LEDGER range.No Gemini or GPT-5.2 primary cell changed under recoding; DeepSeek changed by at most 1 of 15 in E1/E2 and 3 of 15 in E3.
  • Sampler plausibility: NATIVE and NONE failed the plausibility screen through near-universal horizon exhaustion, while LEDGER failed through an almost-all-compromise outcome mix.The Wilson lower bound was 0.80 per cell and 0.97 pooled for NATIVE and NONE; pooled Gemini LEDGER compromise-rate lower bound was 0.82.

7 WHAT THE CONTROLS RULE OUT

The controls rule out extra writing space, generic memory, temperature, concession labels, threshold choices, and model effort as explanations for the LEDGER gap. Hand-coded policies reach agreement under the same engine, indicating that the standing-authority deadline does not inherently prevent bargaining success.

  • Output space, memory, and reflection: 15 of 15 primary cells still fail under a 1024-token NONE rerun, showing that extra output space does not open negotiation.Entropy remains near zero.
  • Output space, memory, and reflection: 15 of 15 runs recur with each of three generic five-slot notebooks and in a GPT-4.1 version, so generic memory does not reproduce the LEDGER effect.The alternatives cover generic state, process notes, and message form.
  • Randomness, thresholds, and effort: Temperature changes nothing from 0.3 to 1.0, while removing concession vocabulary from a relabeled ledger remains compromise-heavy.Nine compromise×consensus threshold pairs likewise change no conclusion.
  • The scripted non-LLM baseline: 14 of 15 games close with a time-based conceder, while a noisy variant and mixed population each close 13 of 15 under the identical engine and classifier.These hand-coded policies show that agreement is achievable without language models and that outcomes track strategy.

8 INTERPRETATION

The interpretation identifies a shared outcome floor in ledger-free configurations: reasoning can make behavior look richer without producing agreement, while negotiation-structured memory breaks the pattern. The ledger works by making relational progress legible, after controls rule out alternative explanations.

  • Interpretation: Every ledger-free configuration shares a collapsed outcome floor, whereas a five-field notebook reliably breaks that pattern.Reasoning restores surface richness without changing the floor.
  • The floor is a default: Agreement requires sustained concessions from at least two parties late in the negotiation, but turn-by-turn defensible COUNTER moves compound into repetition until the deadline triggers fallback.Every LEDGER cell reaches agreement, showing the threshold is reachable but demanding.
  • What reasoning changes: 225 of 225 genuine COUNTER turns make GPT-5.2 NATIVE nearly a textbook solver, while DeepSeek NATIVE explores positions without crossing the commitment threshold.Provider reasoning rearranges behavior within the floor rather than moving runs off it.
  • Why the ledger works, and what to do: Controls rule out token budget, generic memory, randomness, label vocabulary, and information access, leaving self-written state as the distinguishing addition.The five fields plausibly externalize bookkeeping human negotiators use to recognize when a deal is close; the passage flags this as interpretation, not mechanism.

9 LIMITATIONS

The study characterizes deployed provider configurations rather than reasoning as a mechanism and tests a narrow protocol family. Breadth beyond the shared five-actor setup and the single-anchor isolation controls remains untested.

  • Behavioral scope: Provider reasoning is partly hidden and differs across vendors, so the study characterizes three deployed configurations rather than reasoning as a mechanism.A dose–response ladder is available for only one provider, Gemini.
  • Behavioral scope: All environments use one five-actor cast, one vocabulary, and one classifier, limiting protocol-family breadth.Domain changes and scenario variants argue against narrow overfitting, but broader testing is still needed.
  • Behavioral scope: The isolation controls were run on a single anchor, leaving their generality untested.The passage identifies breadth beyond this protocol family as untested.

10 CONCLUSION · A PROMPT TEMPLATES AND THE LEDGER · B PARSER AND OUTCOME CLASSIFIER, IN FULL

Across three environments and four model families, negotiation-structured memory was necessary to move simulations beyond near-universal nonagreement, while provider reasoning changed surface richness but not endings. The released protocol specifies a five-field private ledger, strict action parsing, and deterministic outcome and concession-arc measurement.

  • 10 CONCLUSION: 314 of 315 configurations without negotiation-structured memory ended without agreement, while provider reasoning changed endings not at all but restored measured surface richness.The comparison covered three institutional negotiation environments and four model families.
  • 10 CONCLUSION: A five-field ledger, and nothing else tested, moved models off the nonagreement floor under matched budgets, memory structures, temperatures, and classifier thresholds.The conclusion identifies the ledger as the sole tested intervention that changed this outcome pattern.
  • A PROMPT TEMPLATES AND THE LEDGER: All conditions use the same agent profiles, scenario context, and fixed 15-turn status line, after which unresolved issues go to the regulatory authority.Agent profiles contain names, organizations, ranked objectives, and red lines.
  • A PROMPT TEMPLATES AND THE LEDGER: Each action must be one of SUPPORT, OPPOSE, CONCEDE, COUNTER, or EXIT, paired with a rationale in the prompt format.The action set is closed and explicitly represented in the template.
  • A PROMPT TEMPLATES AND THE LEDGER: The ledger prepends each agent’s private state and requests five capped fields—internal monologue, concessions, current state, opponent assessment, and open issues—plus an action and rationale.The fields are overwritten each turn and never shown to other agents.
  • B PARSER AND OUTCOME CLASSIFIER, IN FULL: Invalid or missing actions default to OPPOSE with provenance tags, while missing ledger fields fall back to the previous turn’s state and every turn’s tag is persisted.Provider errors and empty responses receive separate tags.
  • B PARSER AND OUTCOME CLASSIFIER, IN FULL: A strict parse success requires every turn to contain ACTION:, RATIONALE:, and all five ledger fields; outcomes are then classified deterministically from the last 10 turns.Priority rules identify breakdown, consensus, compromise, authority decision, or deadlock; concession arcs require an earlier OPPOSE/COUNTER followed by later CONCEDE/SUPPORT.

C DATA INTEGRITY AND COLLECTION DATES · D PROVENANCE OF THE NATIVE CELLS · E THE CONTROLS AT A GLANCE

The paper distinguishes clean within-batch comparisons from cross-batch contrasts, documents and replaces an operationally failed GPT-5.2 E3 batch, and clarifies how action provenance and controls should be interpreted. Native-cell parsing metrics measure different properties, while controls use the same evaluation pipeline to test rival explanations.

  • C DATA INTEGRITY AND COLLECTION DATES: Primary data and OpenAI extensions were collected in April 2026, while controls, the matched-budget rerun, free-form arm, and effort ladder were collected in April or July 2026.Within-batch contrasts avoid provider drift; cross-batch comparisons are labeled, categorical outcomes are drift-stable, and finer process claims rely on within-batch cells.
  • C DATA INTEGRITY AND COLLECTION DATES: 225 per cell provider-error responses made the first GPT-5.2 E3 batch unusable across NONE, LEDGER, and NATIVE.Every turn of every run returned a provider error, so the resulting action streams consisted entirely of fallbacks.
  • C DATA INTEGRITY AND COLLECTION DATES: 100% genuine actions characterize the clean full E3 OpenAI-extension rerun, which supplies all reported GPT-5.2 and GPT-4.1 E3 numbers.The failed batch was discarded as an instrument failure and replaced by the rerun.
  • D PROVENANCE OF THE NATIVE CELLS: “Genuine” actions are parsed from model output, whereas “fallback” actions insert OPPOSE and “provider” actions reflect API errors or empty responses.The provenance table recomputes these categories per turn from raw logs; malformed rationales can coexist with genuine actions.
  • D PROVENANCE OF THE NATIVE CELLS: 0.27 strict parse and 0.94 genuine actions for Gemini E1 NATIVE show that strict parsing measures protocol discipline, not action provenance.The strict flag requires every field of every turn to be well-formed, so one malformed rationale can invalidate an entire run.
  • D PROVENANCE OF THE NATIVE CELLS: 0.87 strict parse, 99.1% genuine actions, and 15 of 15 without agreement characterize Gemini E3 NATIVE.The example separates high action genuineness from full-run agreement outcomes.
  • E THE CONTROLS AT A GLANCE: All controls use the same protocol, parser, classifier, and measurements as the main matrix.Table 4 frames each control as removing one rival explanation for the gap between LEDGER and every other condition.

F STATISTICAL DETAIL … J TRANSCRIPT EXCERPTS

The statistical analyses show that LEDGER is the only condition producing agreement, while some NATIVE conditions achieve comparable diversity without agreement. Error exclusion, scripted policies, subtype coding, and transcript excerpts distinguish genuine bargaining, deadlock, and protocol degradation.

  • F STATISTICAL DETAIL: 15/15 Gemini runs and 14/15 DeepSeek runs reached agreement with LEDGER versus 0/15 with NATIVE in E1, with Fisher p-values of 6.5 × 10−9 and 1.0 × 10−7.GPT-5.2 reached agreement in 5/15 LEDGER runs versus 0/15 NATIVE runs, p = 0.021; the six primary-family contrasts survive whole-paper multiplicity correction.
  • F STATISTICAL DETAIL: 90/90 Gemini effort-ladder runs ended with authority decisions, even as high reasoning produced uniqueness of 15/15 in both E1 and E3.The high-level values are operational rather than behavioral because fallback OPPOSE mixing with genuine COUNTER contaminated the measure, and worst-case recoding flips 14 of 15 high-level runs.
  • F STATISTICAL DETAIL: 15/15 read-only state-digest and free-form scratchpad runs ended in authority decisions, but scratchpads increased Gemini entropy to 0.13 and uniqueness to 7/15 from 0.00 and 1/15.The read-only arm contained 225 COUNTER turns, whereas the scratchpad arm pooled 209 COUNTER, 15 SUPPORT, and 1 CONCEDE actions; both differed from E1 LEDGER at p = 6.5 × 10−9.
  • G ERROR-EXCLUDED REANALYSIS: 2 to 15 runs per cell remained after excluding any provider-error or format-error turn, and the substantive pattern was unchanged: only LEDGER cells produced agreement.NONE and NATIVE cells ended without agreement in every retained run, including Gemini E1 NATIVE 4 of 4, DeepSeek E1 NATIVE 7 of 7, and DeepSeek E3 NATIVE 5 of 5.
  • I SUBTYPE CODING AND TRAJECTORY UNIQUENESS: 14/15 DeepSeek NATIVE trajectories were unique, while active unresolved bargaining occurred in 2, 1, and 4 Gemini NATIVE runs across E1–E3 and 10, 11, and 14 DeepSeek runs.Every no-agreement run was coded as active bargaining when it contained genuine SUPPORT or CONCEDE turns, and as counter-repetition deadlock otherwise; GPT-5.2 NATIVE had 0 active runs in every environment.
  • J TRANSCRIPT EXCERPTS: 225/225 GPT-5.2 NATIVE turns in E1 were COUNTER, whereas a Gemini LEDGER excerpt reached compromise on turn 11 through concessions on turns 7 and 11.The excerpts illustrate hard bargaining that never moves, genuine but unresolved bargaining, and the concession cascade associated with LEDGER closure.
Loading 2604.11840v3…