PaperlayerView exact revision on arXiv

Source-linked AI summary

Narrowing the Collaboration Gap, Probably

Mirah Shi, Marcel Hussing, Ira Globus-Harris, Natalie Collina, Aaron Roth, Surbhi Goel

arXiv:2610.19215v1cs.LGcs.AIcs.MA

TL;DR

Agents with private information need concise messages that support aggregation, but proposed actions can omit decision-relevant uncertainty. The paper analyzes and tests probability communication, finding it useful when forecasts are calibrated and showing that post-hoc calibration improves collaboration when probability reports are biased.

  • Problem

    In multi-agent tasks with private information, it remains unclear what concise messages best help agents aggregate decision-relevant information.

    Source: §1 · p. 1

  • Method

    The paper compares action and probability communication in a chained-parity model and maze-solving experiments, and applies post-hoc calibration to probability reports.

    Source: §1.1 · p. 3, §4 · p. 7, §5.1 · p. 9

  • Results

    Probability communication helps when uncertainty is calibrated; post-hoc calibration improves maze success, with mean success rates at least doubling after four rounds versus the base verbalized-probability protocol.

    Source: §1.1 · p. 3, §5.2 · p. 11

  • Takeaways & Limitations

    Calibrated probability messages can give collaborating agents a reliable object to update on, while biased reports can remove their advantage.

    Source: §1.1 · p. 3

    Calibration cannot add maze-specific knowledge absent from communicated probabilities, and its benefit is limited when those reports carry little information.

    Source: §5.2 · p. 11

Abstract

from arXiv · show

Large language models are increasingly deployed as teams of agents that hold different private information about a shared task. How should such agents concisely communicate in a way that promotes aggregation of decision-relevant information? Communicating only proposed actions can discard decision-relevant uncertainty: two agents may prefer the same action for different reasons. We study an alternative interface in which agents communicate calibrated beliefs over actions. We first analyze a simple synthetic setting that captures the structure of multi-round reasoning under partial information. In this setting, we show theoretically and empirically that communicating unbiased probabilities can be strictly more powerful than communicating actions. We then test the same principle in a collaborative maze-solving task with trained transformers and pretrained LLM agents. Our results demonstrate that probability communication outperforms action communication when the exchanged predictions preserve calibrated uncertainty. When communicated probabilities are biased, we show that the benefits of probability communication can disappear. We give a post-hoc conversational calibration intervention that consistently improves decision-making by correcting these biases both in a toy transformer setting and when pretrained LLMs use the corrections in context. These results suggest that probabilistic communication is useful for multi-agent collaboration not merely because it transmits more information, but because calibrated uncertainty gives other agents a reliable object to update on.

1 Introduction

Agents with partial, private information need concise messages that help them combine evidence, but action-only messages can omit decision-relevant uncertainty. The paper compares action and probability communication, then tests whether calibrated beliefs improve collaboration in theory and maze-solving experiments.

1 Introduction

  • Multi-agent collaboration is difficult when bandwidth, context length, modality differences, or limits on processing make full information sharing infeasible.Prior work reports failures in information sharing, belief updating, and joint decision-making under partial information [Li et al., 2023, Wilie et al., 2024] [Davidson et al., 2025, Eisenstein et al., 2026].Source: §1 · p. 1
  • Action-only messages can discard decision-relevant uncertainty, while agents may prefer the same action for different reasons.The agreement literature motivates exchanging beliefs so agents with different observations can aggregate information.Source: §1 · p. 2

1.1 Our contributions

  • The paper compares action-only messages with probability messages, including verbalized probabilities and empirical distributions from independent LLM response rollouts.The study tests the hypothesis that communicating calibrated beliefs supports collaboration more effectively than communicating only preferred decisions.Source: §1.1 · pp. 2–3
  • In chained parity, exact probabilities aggregate one new piece of information per round, while action messages stall; in maze experiments, probability communication helps when uncertainty is calibrated.Biased probability reports can erase the advantage, while post-hoc calibration recovers performance in transformer and pretrained LLM experiments.Source: §1.1 · p. 3

2 Preliminaries

The setup compares agents exchanging proposed actions with agents exchanging probability distributions over actions across multiple rounds. Calibration describes whether those probabilities match outcomes, while cross-calibration asks whether that remains true after conditioning on another communicated forecast.

2 Preliminaries

  • Two agents observe separate parts of a feature, and the correct action depends on their combined information.Neither agent can reliably identify the correct action from its own observation alone.Source: §2 · p. 3
  • Agents alternate messages for a fixed number of rounds, with each message much smaller than the speaker’s private observation.After the final round, they execute the final action message or the most probable action in the final probability message.Source: §2 · p. 3
  • Action communication sends one proposed action; probability communication sends a distribution over actions.The protocol executes the final proposed action or the argmax of the final distribution, respectively.Source: §2 · p. 3
  • A forecast is calibrated when its stated probabilities match the long-run frequencies of correct actions among instances with that forecast.Cross-calibration requires a forecast to remain calibrated after conditioning on another communicated forecast, not on the other agent’s private observation.Source: §2 · pp. 3–4
  • Conversation calibration requires the initial forecast to be calibrated and each later forecast to be cross-calibrated with respect to the previous round’s forecast.This protocol-specific condition applies through round K.Source: §2 · p. 4

3 When do probabilities outperform actions?

In the chained-parity model, probability messages preserve a sufficient summary of accumulated evidence, whereas action messages can collapse distinct evidence states. Under the distinct-partial-sums condition, probability exchange reaches the full-information posterior; the action channel can transmit only the first bit.

3 When do probabilities outperform actions?

  • The chained-parity task models multi-round reasoning as a sequence of intermediate states, each advanced by new evidence that neither agent observes alone.Alice and Bob observe alternating bits, and the target posterior depends on the full bit vector.Source: §3 · pp. 4–5
  • A scalar probability message can identify the accumulated prefix parities when distinct bit prefixes yield distinct partial sums.The next speaker combines that prefix with their newly observed bit to determine the next parity, so the probability exchange advances one message at a time.Source: §3 · pp. 5–6
  • Under Assumption 3.1, probability exchange reaches pK = p⋆ and attains the full-information Bayes risk after K rounds.At round k, its message is pk = 1/2 + sum over j ≤ k of αjSj; the excess squared loss reflects the unrevealed tail before the final round.Source: §3 · p. 6
  • Action exchange repeats the first bit in every round, transmitting no later bit; for K = 3 and α = (0.20, 0.15, 0.10), its decision disagrees with the full-information Bayes action with probability 1/4.The action message retains only the sign of the posterior, which can collapse later evidence states to the same first action.Source: §3 · p. 6
  • In trained-transformer experiments, learned probability messages track the theoretical loss curve, while learned action messages stall as predicted.Figure 3 compares squared loss over five rounds and reports that learned probability communication attains the theoretical optimum.Source: Figure 3 · p. 6, §3 · p. 6

4 When do probability reports need calibration?

Biased probability reports can undermine communication, but post-hoc calibration recovers ideal forecasts under stated conditions. The section describes an iterative calibration procedure and reports recovery in maze results.

4 When do probability reports need calibration?

  • Global confidence distortion makes reports too extreme or mild, while sycophantic anchoring pulls them toward the previous report regardless of evidence.The model uses c to control confidence and λ to control anchoring; raw reports may therefore be unreliable for the next agent to update on.Source: §4 · p. 7
  • With an ideal previous posterior, a sufficiently expressive predictor trained with a strictly proper loss recovers the posterior and conversation-calibrated forecasts for any allowed distortion parameters.Applied inductively, this recovers the prediction guarantee; the result assumes the predictor class contains the conditional label distribution.Source: §4 · p. 7
  • The recovery result extends beyond the affine example to distortions invertible in the current posterior when the previous report is fixed.If a receiver treats a biased report as valid and projects it to a possible message, the first bit may decode correctly while the second can flip.Source: §4 · p. 7
  • Each round, an agent’s raw forecast is recalibrated using labelled transcripts and the previous calibrated message before it is sent onward.The post-hoc map takes the raw forecast and previous message as inputs.Source: §4 · pp. 7–8
  • Action performance stagnates while probability communication improves gradually; with biased reports, miscalibration and anchoring worsen performance, while recalibration recovers some or nearly all of it.Self-calibration partially recovers performance under global miscalibration; conversation calibration recovers nearly all baseline performance when both biases are present.Source: Figure 5 · p. 8, Figure 4 · p. 8

5 Maze experiments with learned and pretrained agents

The maze experiments test whether communicating probability distributions helps agents combine partial information, using trained transformers and pretrained LLMs. Probability exchange is most useful when calibration makes communicated uncertainty reliable; biased probabilities can lose that advantage.

5 Maze experiments with learned and pretrained agents

  • The maze task gives each agent half of a partially hidden map, so the pair must combine observations over multiple rounds to choose a correct move.Experiments use 6 × 6 mazes with 30% wall density and a single correct solution.Source: §5 · pp. 8–9

5.1 Controlled transformer experiments

  • Across four communication rounds, trained transformers using probabilities outperformed action communicators, with success improving consistently only under probability exchange.The task trains small transformers to predict the next move from a partial-view maze and prior messages.Source: §5.1 · p. 9
  • With global miscalibration and sycophantic anchoring, conversation calibration recovered nearly all unbiased-baseline performance, while self-calibration recovered only part of the loss.Conversation calibration uses current and previous-round probabilities; self-calibration uses only the current probabilities.Source: §5.1 · p. 9

5.2 Pretrained LLM experiments

  • After four rounds, post-hoc calibration at least doubled mean maze success rates relative to the base verbalized-probability protocol across the tested models.Calibration also significantly decreased calibration loss for all four models.Source: §5.2 · p. 11
  • For GPT-5.4-nano, calibrator gains were similar to a rare-action correction, while on GPT-5 the calibrator slightly outperformed that baseline.The baseline replaces predictions favoring rare actions with the marginal distribution over down and right.Source: Figure 7 · p. 11, §5.2 · p. 12
  • Post-hoc calibration matched or outperformed sampled probabilities: it was within variance of sampling on GPT-4.1-mini and achieved higher mean success on GPT-5.4-nano.The calibration procedure requires one rollout, whereas sampled probabilities use multiple responses.Source: §5.2 · p. 12, Figure 8 · p. 12
  • Unrestricted full-response communication performed poorly, with success close to zero across rounds for both tested models, despite GPT-5.4-nano solving over 85% with the full maze.The comparison suggests that providing more information does not necessarily improve collaboration.Source: §5.2 · pp. 13–14, Figure 10 · p. 13

A Additional Related Work

  • The study connects calibrated forecasting and agreement work with LLM uncertainty elicitation, sampled uncertainty, multi-agent collaboration, and learned communication, while focusing on interpretable beliefs over actions.Unlike debate or learned-protocol work, the agents have different private observations and the question is which low-bandwidth message preserves decision-relevant information.Source: A Additional Related Work · pp. 18–19

B Warm-Up Example for Section 3

The warm-up shows that action messages can collapse distinct evidence states, while probabilities can preserve the distinction and let an agent combine private information. The proof passages state the conditional-independence lemma used in the later chained-parity analysis.

B Warm-Up Example for Section 3

  • Alice sends a1 for either observed bit, so her action message carries no information about y1.The two observations induce the same best action even though the underlying evidence states differ.Source: B Warm-Up Example for Section 3 · p. 19
  • Probability communication lets Bob identify the full state and choose the correct action with probability one, while action-only communication leaves the exceptional state unidentified.Action messages yield a1 with prior probability 3/4 + ϵ; distinct posterior probabilities reveal Alice’s bit.Source: B Warm-Up Example for Section 3 · pp. 19–20
  • The separation concerns whether messages preserve distinct posterior states, not whether they are syntactically actions or probabilities..Source: B Warm-Up Example for Section 3 · p. 20

C Proofs for Section 3

  • Conditional vanishing: an independent mean-zero bit makes the conditional expectation of its product with a function of the other bits equal zero.The lemma also applies to a product over a set containing such an independent bit.Source: C Proofs for Section 3 · p. 20

C.1 Proof of Theorem 3.2

In the chained-parity proof, probability messages accumulate prefix-parity information until they recover the full-information posterior. Action messages can instead stall after revealing only the first bit, while exact recovery may require all rounds even when low excess loss is achievable earlier.

C.1 Proof of Theorem 3.2

  • After K rounds, probability exchange reaches p⋆ and the full-information Bayes risk.The round-k message is 1/2 + the sum of revealed weighted prefix parities.Source: §C.1 · p. 20
  • Each round’s probability message identifies the revealed prefix parities, allowing the speaker to add the next parity while later unseen terms have conditional mean zero.This argument uses the distinct-partial-sums condition to recover the prefix from the previous scalar message.Source: §C.1 · p. 20
  • Without distinct partial sums, previous scalar messages still reduce risk each round, but the closed-form per-round improvement is lost; sending the full history restores it.With the full history, the improvement is α_k^2.Source: §C.1 · p. 22
  • Low excess squared loss can require fewer than K rounds, but exact recovery of h⋆ may require all K because the unrevealed tail can flip its sign.The stopping condition is that the remaining sum of squared weights is at most ε.Source: §C.1 · p. 22

C.2 Proof of Theorem 3.3

  • Action messages equal the first bit at every round, so they do not change the induced decision after round one.For the stated K = 3 instance, the action decision disagrees with h⋆ on an event of probability 1/4 at every round.Source: §C.2 · p. 22

D Proofs for Section 4

The proofs show how post-hoc calibration can recover an ideal posterior from biased reports, and how a receiver using nearest-valid projection can instead decode a later message incorrectly. The projection failure depends on the bias and anchoring terms.

D Proofs for Section 4

  • A predictor trained on suitable features and a strictly proper loss recovers the conditional label probabilities, yielding a forecast calibrated even conditional on those features.When the features include a previous forecast, the learned forecast is also cross-calibrated with respect to it.Source: D Proofs for Section 4 · pp. 22–23
  • When λ = 0 and c > 0, self-calibration recovers p_k from the raw report, whereas using the raw report directly incurs excess squared loss.The raw report is an invertible transformation of the ideal scalar in this case.Source: D Proofs for Section 4 · p. 23
  • With the previous ideal message available, post-hoc cross-calibration recovers p_k despite confidence distortion or anchoring in the raw report.The chained-parity result applies for c > 0 and any λ, subject to the stated model conditions.Source: D Proofs for Section 4 · pp. 23–24
  • Nearest-valid projection decodes round one correctly for c > 0, but round-two anchoring can cause it to decode the next bit incorrectly.The round-two wrong-decode event occurs when |c + λ − 1|α1 > cα2; equality is a tie.Source: D Proofs for Section 4 · p. 24
  • A round-two wrong decode adds squared-loss excess 4α2^2 conditional on the error, or 2α2^2 averaged over b2 after correct round-one decoding.The projection failure disappears when c + λ = 1, while true calibration of the affine report requires c = 1 and λ = 0.Source: D Proofs for Section 4 · pp. 24–25

E.1 Task design

The experimental details specify the chained-parity and maze datasets, then describe the transformer inputs and round-by-round training procedure used to pass messages between agents.

E.1 Task design

  • Chained-parity data use 30-bit strings and coefficients summing to 0.49, with alternating disjoint bits, two million examples, and a 90/10 train-test split.Labels are sampled from one-half plus the weighted sum of prefix products.Source: §E.1 · p. 25
  • Maze data use 6 × 6 single-solution mazes with 30% wall density, disjoint random half-cell views, and two million step-level examples split 90/10.Each label is the correct next move along the unique solution path.Source: §E.1 · p. 25

E.2 Transformer experiments

  • For both tasks, small character-level GPT transformers predict answers from serialized inputs, with each agent’s private view and the current position encoded in its input.The maze success rate counts mazes for which every move on the solution path is predicted correctly.Source: §E.2 · pp. 25–26
  • Agents communicate through appended messages across rounds, with a freshly initialized transformer trained each round so performance gains must come through those messages.The training loop alternates speakers and appends each message to the next agent’s inputs.Source: §E.2 · p. 26

E.3 LLM experiments

The LLM experiments compare communication protocols in a two-agent collaborative maze task, while the appendix specifies the prompts and conversation-calibration loss used in the experiments.

E.3 LLM experiments

  • LLM agents receive their own masked 6 × 6 maze, the move history, and the partner’s latest response, then choose one move per conversation.The experiments use four rounds per maze and evaluate 30 held-out mazes with 10 rollouts each; an incorrect move ends the maze.Source: §E.3 · p. 26

E.3.1 Prompts

  • Prompts request a move and, in verbalized settings, a probability vector over down, right, up, and left whose entries sum to 1.The prompt frames the task as jointly solving a maze from maps with hidden cells; the solo baseline instead supplies the full maze.Source: §E.3.1 · pp. 27–28

E.3.2 Conversation Calibration Loss

  • Conversation calibration loss compares bin-averaged predictions with empirical outcomes within cross-bins formed from the model’s and collaborator’s distributions.The method uses hard binning and reports the loss with B = 10; at round 1, it reduces to standard self-calibration loss.Source: §E.3.2 · pp. 28–29

F Examples of Multi-Round Communication in Mazes

The maze examples illustrate how iterative probability exchange can resolve uncertainty about a move, including a case where one agent corrects the other’s initially mistaken preference.

F Examples of Multi-Round Communication in Mazes

  • The examples track each agent’s partially observed maze and probabilistic predictions over four rounds, with the correct answer shaded.Agent A communicates on odd rounds and Agent B on even rounds.Source: Figure 12 · p. 29, F Examples of Multi-Round… · p. 29
  • 78% probability was assigned to the correct move right in the first round; in the second example, Agent B assigned 89% to down and Agent A reached 99% for down.The first example’s prediction is progressively reinforced over four rounds; other moves in these mazes require only two rounds because one agent knows the answer exactly.Source: F Examples of Multi-Round… · pp. 29–30, Figure 12 · p. 29

G Limitations

The limitations section bounds the paper’s claims to one aspect of collaboration and identifies unresolved theory and evaluation scope.

G Limitations

  • The work studies communication protocols for two agents with partitioned information, not the full complexity of multi-agent collaboration.The authors identify task decomposition, trust, prompt sensitivity, general conditions for the action–probability gap, realistic tasks, and systems with more than two agents as unaddressed or open areas.Source: G Limitations · p. 30

H Compute Resources

The section reports the computing resources and approximate GPU hours used for the experiments.

H Compute Resources

  • The experiments used approximately 300–400 GPU hours; maze-transformer training took about 6–12 hours per run, depending on the GPU.A single parity experiment could be trained on a household-grade GPU in under an hour; LLM experiments used OpenAI’s public API and required no local resources beyond a standard laptop.Source: H Compute Resources · p. 30

Related papers

Loading 2610.19215v1…