PaperlayerView exact revision on arXiv

Source-linked AI summary

Partially Observable Zero-shot coordination by Predicting Intention of Partner

Jinnyeong Yang, Yuhwan Jeong, Hoyong Kwon, Minseok Kim, Jihun Kim, Kuk-Jin Yoon

arXiv:2610.08142v1cs.AIcs.MA

TL;DR

In embodied collaboration, limited sensing and movement can leave agents without full information about their partner and task state. PIP combines training-time partner representation learning with belief networks, and achieves the highest mean performance among compared methods across three benchmarks.

  • Problem

    In embodied settings, limited sensing and agent movement prevent full observability of task and partner states, leaving collaborators to act with incomplete information about partner behavior and task state.

    Source: §1 · p. 1

  • Method

    PIP distills a teacher representation from both agents’ local views into an ego-view Joint-view VAE student and uses Partner-state Belief networks to infer hidden partner information.

    Source: §1 · p. 2

  • Results

    PIP achieves the highest mean performance across Burrito-PO, Overcooked-PO, and a Melting Pot substrate, with human evaluation and diagnostic analyses supporting its design.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    The results support the importance of partner representations and belief-aware decision making for zero-shot coordination under spatially restricted observations.

    Source: §7 · p. 9

    The partner’s strategy is not directly observed and must be inferred from local interaction history alone.

    Source: §3 · p. 3

Abstract

from arXiv · show

Zero-shot coordination in embodied settings requires acting while the partner is intermittently out of view, leaving existing methods with ambiguous partner representations and uncertainty over hidden partner states. We propose Predicting Intention of Partner (PIP) to jointly address these challenges. PIP uses a Joint-view VAE to distill richer training-time evidence from the union of both agents' local observations into a partner representation available from local observations alone. Partner-state Belief networks further infer the partner's hidden location and behavioral tendencies from the ego agent's interaction history. We evaluate PIP in Burrito-PO, Overcooked-PO, and a Melting Pot substrate, together with a human evaluation in Burrito-PO. PIP attains the highest mean performance among the compared methods across all three benchmarks. Human evaluation and diagnostic analyses further support coordination with unseen partners and the contributions of both components under partner occlusion.

1 Introduction

Zero-shot coordination in embodied settings must handle unseen partners when limited sensing and movement make partner and task states only partially observable. PIP combines a Joint-view VAE for partner generation with Partner-state Belief networks for hidden-state inference.

1 Introduction

  • Limited sensing and agent movement can hide task and partner states, leaving embodied collaborators with incomplete information about partner behavior and task state.Unlike rule-defined hidden information such as in Hanabi, embodied partial observability is spatial and dynamic.Source: §1 · p. 1
  • Existing spatial-PO ZSC studies have focused on environment design or symmetry, while viewpoint-induced failure modes remain underexplored.The paper presents its work as the first to address these failures in both partner generation and policy learning.Source: §1 · p. 2
  • The Joint-view VAE distills a teacher posterior from both agents’ local observations into an ego-view student, preserving behaviorally meaningful partner variation under partial observability.The joint view is training-time evidence from two local views, not access to the global state.Source: §1 · p. 2
  • PIP addresses two challenges: local views can collapse distinct partner behaviors into overlapping latent regions, and occlusion hides partner location, recent actions, and intended subtasks.The first challenge motivates richer training-time evidence; the second motivates inference from the ego agent’s history.Source: §1 · p. 2
  • Partner-state Belief networks estimate an out-of-view partner’s location and likely behavior from the ego agent’s interaction history.Logged partner information supervises belief heads during PPO updates but is not policy input at deployment.Source: §1 · p. 2
  • PIP is evaluated on Overcooked-PO, Burrito-PO, and Melting Pot, with human evaluation and analyses supporting coordination with unseen partners and component contributions.The introduction reports extensive experiments across the three benchmarks.Source: §1 · p. 2

2 Related works

Prior ZSC work improves coordination with unseen partners through diverse populations, convention-robust training, and generative behavior models. Under partial observability, spatially changing viewpoints leave intermittent partner loss less explicitly addressed than rule-defined hidden information.

Zero-Shot Coordination and Partner Diversity

  • Partner-diversity approaches include population training, diversity optimization, and generative latent spaces that expose policies to varied behaviors and conventions.Other-Play, Any-Play, CEC, GAMMA, TALENTS, GOAT, and E3T represent different strategies in this landscape.Source: §2 · p. 3

Zero-Shot Coordination under Partial Observability

  • Hanabi-style partial observability hides information prescribed by game rules, while prior methods combine novel-partner compatibility with inference over that hidden information.ERS broadens the symmetry class in Dec-POMDPs.Source: §2 · p. 3
  • In spatial coordination, visibility changes with agent movement; prior work evaluates restricted-view environments, but intermittent loss of partner observations is handled only implicitly.Explicitly inferring currently unobserved partners’ positions and actions alongside incomplete partner representations remains limited.Source: §2 · p. 3

3 Preliminaries

The preliminaries frame decentralized coordination as a two-player partially observable process and describe latent-variable methods for representing partner strategies. Policy execution uses local interaction history, with logged partner information reserved for auxiliary belief supervision.

3 Preliminaries

  • A two-player partially observable Markov decision process has incomplete information, local observations, transition dynamics, shared reward, and discounted return.The environment tuple includes state, action and observation spaces, transition and observation functions, reward, and discount factor.Source: §3 · p. 3
  • Generative latent-variable ZSC encodes partner trajectories into a latent strategy and decodes that strategy into predictions of the partner’s next action.The latent can capture convention, preference, or execution style; training uses a variational objective and a prior, after which sampling the latent instantiates partner policies.Source: §3 · pp. 3–4
  • During decentralized execution, the policy uses only ego-centric observation history; logged partner-state information is auxiliary supervision for belief heads during PPO training.The partner’s strategy is not directly observed and must be inferred from local interaction history alone.Source: §3 · pp. 3–4

4 Proposed method

PIP combines a Joint-view VAE for partner-behavior modeling with a Partner-state Belief policy for decision-making under hidden partner states. The method distills richer training-time evidence into a local-view partner generator and uses interaction history to maintain beliefs about partner behavior and location.

4 Proposed method

  • PIP combines a Joint-view VAE for partner generation with a Partner-state Belief policy for learning from incomplete observations.Together, the components address partner generation and policy learning under hidden partner states.Source: §4 · p. 4

4.1 Joint-view VAE

  • The Joint-view VAE uses a teacher posterior from richer joint observations to guide a student conditioned on the agent-centered partial observation.The union view includes cells visible to either agent, while cells invisible to both remain masked.Source: §4.1 · p. 4
  • Training first fits a teacher VAE, then freezes it while training the student with its negative ELBO and a posterior-distillation term.The trained student decoder is used as a deployment-compatible partner generator during target-policy training.Source: §4.1 · pp. 4–5

4.2 Partner-State Belief Policy

  • The Partner-state Belief module maintains a separate belief state from local observation history to track partner-related information hidden at deployment.The observation history is partly masked to remove a direct orientation shortcut and encourage inference from remaining evidence.Source: §4.2 · p. 5
  • Behavioral Belief separates predictions of short-term partner reactions from predictions of more persistent behavioral tendencies.The two behavior heads are trained with soft cross-entropy losses.Source: §4.2 · p. 5
  • Location Belief predicts soft distributions over coarse partner-location cells at current and future offsets, mixing future-cell targets with reachability information.When the partner has been unobserved for more than Hmax steps, the reachability prior is replaced by a uniform spatial prior.Source: §4.2 · p. 5
  • The actor receives the belief through an additive interface, while PPO gradients are blocked from directly reshaping the belief RNN.The belief RNN is trained by the behavioral and location objectives, while PPO learns how to use its injected message for reward maximization.Source: §4.2 · p. 6

5 Experiments

Experiments evaluate PIP against held-out partners across three partially observable benchmarks, with a human study in Burrito-PO. PIP leads the compared methods in each benchmark, and human participants also favor it on reported coordination measures.

5 Experiments

  • Policies, partner populations, trajectory datasets, and generative models were trained separately for each environment; deployed agents used only local observations.The joint-view teacher used both agents’ local observations during VAE training, while the student and deployed policies remained restricted to local observations.Source: §5 · p. 6

5.3 Melting Pot

  • PIP achieves the best performance among compared methods in Melting Pot, with results alongside Overcooked-PO in Table 2.The authors caution that official bots preclude direct cross-benchmark comparisons.Source: Table 2 · p. 7, §5.3 · p. 7

5.1 Burrito-PO

  • PIP achieves the best performance across all four Burrito-PO layouts, with its relative improvement particularly large on Ring.The Ring route constrains future partner movement, helping the belief module anticipate hidden position and movement direction from local observations.Source: §5.1 · p. 7

5.2 Overcooked-PO

  • PIP achieves the best performance among compared methods on Overcooked-PO CounterCircuit.This evaluation uses a 5 × 5 local observation in a cooking environment where the partner and its current subtask may become unobservable.Source: §5.2 · p. 7

5.4 Human Evaluation

  • Participants earned the highest sparse reward with PIP, ranked it first most often, and gave it the highest score in all six survey categories.The study recruited 66 participants to interact with anonymized GAMMA, GOAT, and PIP agents in randomized order.Source: §5.4 · p. 7, Figure 2 · p. 8

6 Additional Analysis

Ablations support complementary contributions from the distilled VAE and Partner-state Belief components. Representation and occlusion analyses further characterize the latent space and when belief information changes policy behavior.

6.1 Component Ablation

  • Replacing the partial-observation VAE with a student distilled from a joint-view teacher improves performance both without and with Partner-state Belief.The authors interpret this as mitigating representation degradation and providing more useful latent partner representations for generated-partner training.Source: §6.1 · p. 8
  • Combining the distilled VAE and Partner-state Belief achieves the best ablation performance, supporting complementary roles for generated-partner training and partner-state estimation.Behavioral and Location Belief each improve over the distilled VAE alone; Partner-state Belief also improves performance without the distilled VAE.Source: §6.1 · p. 8

6.2 Joint-view VAE Representation Analysis

  • The distilled student VAE outperforms fixed-σ and heteroscedastic baselines on four clustering metrics, with especially large gains on NMI and ARI.It also achieves the highest coverage and lowest MMD²; coverage is evaluated using log qagg(z) on prior samples and squared MMD between qagg and p(z).Source: §6.2 · pp. 8–9, Figure 4 · p. 8

6.3 Partner-state Belief under Occlusion

  • Action-distribution differences between PIP and its belief-lesion policy grow after the partner leaves view, then shrink at large time-since-last-seen values.PIP also consistently outperforms the lesion policy across occlusion groups, with the largest gap for partners occluded most frequently.Source: §6.3 · p. 9, Figure 5 · p. 9, Figure 6 · p. 9

7 Conclusion

PIP combines a Joint-view VAE for deployment-compatible partner representations with Partner-state Belief networks for hidden-state inference. It achieves the highest mean performance across three benchmarks, with human evaluation and diagnostic analyses supporting its design.

7 Conclusion

  • PIP combines a Joint-view VAE for deployment-compatible partner representations with Partner-state Belief networks for hidden-state inference, achieving the highest mean performance across Burrito-PO, Overcooked-PO, and a Melting Pot substrate.Human evaluation and diagnostic analyses further support its design.Source: §7 · p. 9

A Limitations

The belief module requires auxiliary labels for future partner actions and positions, which may be costly to obtain in decentralized real-world settings. Evaluation partners are constructed as held-out behavior-preference agents and selected to retain behavioral diversity.

A Limitations

  • Belief training uses future-action and partner-position labels available in simulation but potentially difficult or costly to obtain in decentralized real-world settings.The authors identify weakly supervised or self-supervised belief learning as future work.Source: A Limitations · p. 13

B.1 Construction of BP Agent Candidates

  • Burrito-PO and Overcooked-PO evaluation pools comprise held-out behavior-preference agents with distinct tendencies, and none of their policies, checkpoints, or reward weights enters target-policy training.Burrito-PO uses 24 candidates per layout; Overcooked-PO draws 60 candidates from admissible preference vectors.Source: §B.1 · p. 13

B.2 Selection of Evaluation BP Agents

  • Twelve partners are selected per layout by maximizing the log-determinant of their behavior-feature similarity submatrix, favoring a compact set of diverse behaviors.Candidate features summarize event counts from five rollouts of each candidate’s corresponding best-response agent.Source: §B.2 · p. 14

B.3 Zero-shot Evaluation Protocol

Burrito-PO and Overcooked-PO evaluations use selected held-out partners, both role assignments, and 120 episodes per evaluation. Melting Pot uses the three official bots in 300 episodes.

B.3 Zero-shot Evaluation Protocol

  • Each Burrito-PO or Overcooked-PO evaluation pairs the coordinator with 12 selected partners under both spawn-position role assignments, collecting five episodes per ordered pairing for 120 total.This protocol yields 10 episodes with each selected partner; role swapping addresses potential asymmetry from ego-centric inputs and agent-specific training signals.Source: §B.3 · p. 15
  • Melting Pot evaluation pairs the target policy with the cooperator, defector, and mixed official bots for 100 episodes each, reporting mean episodic reward and standard error.The total is 300 episodes on the original 27 × 27 coop_mining map.Source: §B.3 · p. 15

C Implementation Details

Implementation uses partial-observation settings and benchmark-specific training data and models; computational costs rise relative to GAMMA/GOAT while remaining around 500 Hz at batch size 64.

C.1 Window Size Selection

  • Burrito-PO uses a 7 × 7 view, Overcooked-PO a 5 × 5 view, and Melting Pot its official egocentric range; remaining hyperparameters follow baselines when possible or are grid-searched.The Burrito-PO window was chosen to retain intermittent visibility without near-zero or near-full visibility.Source: §C.1 · pp. 15–16, §C.2 · p. 16

C.3 Reward Function

  • Burrito rewards share sparse dish-delivery rewards equally, while agent-specific dense shaping rewards aid intermediate progress early and are annealed toward zero.Delivery rewards vary by whether a dish matches the active order, another listed order, or neither.Source: §C.3 · p. 16

C.4.1 Burrito-PO.; C.4.2 Overcooked-PO.; C.4.3 Melting Pot.

  • Generative-partner datasets contain 1,728 Burrito-PO trajectories, 20,000 Overcooked-PO trajectories, and 2,304 Melting Pot trajectories from 24-policy training populations.Burrito-PO and Melting Pot enumerate ordered policy pairs; Overcooked-PO collects 20,000 trajectories.Source: §C.4.1 · p. 17, §C.4.2 · p. 17, §C.4.3 · p. 17

C.5 Baselines

  • Strict-PO baselines adapt observation inputs: FCP and MEP retain their training algorithms, while E3T masks unseen partner actions and GAMMA/GOAT use PO-trained partner VAEs and IPPO cooperators.ERS reduces its population and pair budget for computational reasons while retaining its symmetry architecture and procedure.Source: §C.5 · pp. 17–19

D Computational Resources

  • 591K parameters, 2.05 ms actor inference, and 0.764G FLOPs characterize PIP, versus 463K, 1.13 ms, and 0.381G for the GAMMA/GOAT baseline.The reported inference rate is roughly 500 Hz at rollout batch size 64.Source: Table C.3 · p. 19, D Computational Resources · p. 20

C.6 Policy and VAE model details

  • Policy models use a CNN and recurrent backbone; the target adds a Partner-state Belief branch, and the Joint-view VAE trains a richer-view teacher before distilling its posterior to a local-view student.The student uses KL(qT |qS) so its posterior covers modes captured by the teacher.Source: §C.6 · pp. 19–20

E.1 Burrito-PO

The appendices detail benchmark environments and diagnostics: Burrito layouts vary spatial constraints, Overcooked requires recipe inference, and Melting Pot tests transfer under distinct conditions. Latent-space and belief analyses report partner-specific representations and predictions of hidden partner state.

E.1 Burrito-PO

  • Open, Ring, and Hallway require 137, 156, and 172 solo shortest-path steps respectively, while Forced-Coord enforces spatial separation and handoff-style coordination.The four layouts vary in openness, bottlenecks, object placement, and station separation.Source: §E.1 · pp. 20–21, Figure E.1 · p. 21

E.2 Overcooked-PO

  • The Overcooked-PO counter circuit requires inferring a partner’s recipe from pot contents and dispenser visits because its three recipes use both onions and tomatoes.Its one-tile circular corridor prevents cooks from passing and obscures them when they are on opposite sides.Source: Figure E.2 · p. 21, §E.2 · p. 22

E.3 Melting Pot

  • Melting Pot training uses a reduced 16 × 16 coop_mining map after the original 27 × 27 map caused out-of-memory errors, with zero-shot evaluation on the original map.The reduced map preserves dynamics and reward rules; evaluation uses the official bots.Source: §E.3 · p. 22, Figure E.3 · p. 22

E.4 Discussion of Environments

  • Melting Pot results are interpreted as transfer evidence rather than directly compared in improvement magnitude because its observation interface, task dynamics, and partner construction differ.Burrito-PO is the primary setting for partner-state inference, while Overcooked-PO tests the approach under a different task structure and held-out-partner protocol.Source: §E.4 · pp. 22–23

F.1 Latent-Space Inspection via t-SNE

  • On held-out Burrito partner trajectories, the distilled student forms more separated partner-specific latent clusters than fixed-σ and heteroscedastic encoders despite using the same local observations at test time.The visualization is consistent with higher NMI and ARI reported in the main paper.Source: §F.1 · p. 23, Figure F.1 · p. 23

F.2 Partner-state Belief Prediction Analyses

  • Location Belief outperforms statistical baselines on current coarse location and future location prediction, while reactive action accuracy declines with horizon and tendency predictions remain more stable.Current-location evaluation considers moving partners outside view, grouped by time since last seen; location bins cover 2 × 2 map regions.Source: §F.2 · pp. 23–24, Figure F.2 · p. 24

F.3 Full-Observation Evaluation

The appendix evaluates PIP against fuller-observation training, simplified controls, and test-time observation corruption, and documents the human-study procedures. PIP retains its performance under the tested conditions, while the human study uses anonymized partners and ego-centric partial observations.

F.3 Full-Observation Evaluation

  • Full-observation training changed PIP performance only marginally (+7.2), while GAMMA fell −20.00 and GOAT fell −62.47.The comparison reuses the same held-out partners; the authors hypothesize that GOAT’s degradation primarily reflects a shift in partner behavior distributions.Source: §F.3 · pp. 24–25

F.4 Capacity and Supervision Controls

  • None of the tested capacity and supervision controls matched PIP; capacity alone lowered GAMMA’s mean reward, and supervision controls performed substantially worse.The controls enlarged GAMMA’s policy, added partner-state prediction heads, or combined both changes; their action targets did not exactly match PIP’s belief supervision.Source: §F.4 · p. 25, Table F.2 · p. 25

F.5 Robustness to Observation Corruption

  • At corruption rate r=0.2, PIP returned 207.1 under sparse blackout and 187.8 under noisy corruption, remaining above clean-observation GAMMA at 163.3 ± 10.6.Figure F.3 plots mean return against 12 partners as corruption increases from 0 to 0.2; sparse corruption masks the whole observation, while noisy corruption shuffles channels.Source: §F.5 · pp. 25–26, Figure F.3 · p. 26

G.1 Participant Consent and Data Protection

  • The consent process used anonymous IDs, collected no personal or tracking identifiers, and allowed voluntary participation and withdrawal.The form described the study, data collection and protection, risks, and withdrawal conditions.Source: §G.1 · p. 26, Figure G.1 · p. 27

G.2 Gameplay Interface

  • Participants completed an interactive tutorial and 60 seconds of free play, then played one 120-second episode with each of three anonymized AI policies.Gameplay used ego-centric partial observation, with areas outside the participant’s visible region masked.Source: §G.2 · pp. 26–28, Figure G.2 · p. 27

G.3 Post-Game Evaluation and Preference Survey

  • After each episode, participants rated their AI partner on coordination-related dimensions and then ranked all three partners by collaboration experience.The post-game survey covered coordination quality, predictability, adaptability, role complementarity, subjective experience, and teamwork.Source: §G.3 · pp. 28–29, Figure G.3 · p. 29

G.4 Human Evaluation Details

In a within-subject human evaluation, PIP achieved the highest task reward and was most preferred. Participants also rated it highest across survey categories, with significant differences from both baselines in the reported tests.

Survey Score Construction

  • All participants evaluated all three agents; survey items used 1–7 ratings, with negatively worded items reverse-coded before category means were calculated.The human evaluation analyzed task performance, preference rankings, and post-episode survey scores.Source: §G.4 · p. 29

Task Performance and Preference Ranking

  • PIP’s mean sparse reward was 476.0 versus 355.7 for GAMMA and 370.4 for GOAT; 48 of 66 participants (72.7%) ranked PIP their most preferred partner.The reward was 33.8% above GAMMA and 28.5% above GOAT.Source: §G.4 · pp. 29–30

Statistical Analysis

  • PIP significantly outperformed both baselines on sparse reward and preference rank, with large effect sizes (r ≥0.62) and significance retained after Holm-Bonferroni correction.The tests were within-subject paired Wilcoxon signed-rank tests following Friedman tests.Source: §G.4 · p. 30, Table G.3 · p. 30

Survey Score Decomposition

  • PIP received the highest score in every survey category, with largest relative gains in Overall, Subjective Experience, Predictability, and Adaptability.These differences suggest participants perceived PIP as easier to understand, more responsive, and more desirable to cooperate with under partial observability.Source: §G.4 · p. 30, Table G.4 · p. 30
  • PIP significantly outperformed both baselines in all six survey categories, with large effect sizes throughout (r ≥0.60), and comparisons remained significant after correction.The paired survey comparisons used within-subject nonparametric tests; reported Wilcoxon p-values were uncorrected.Source: Table G.5 · p. 30, §G.4 · p. 31
Loading 2610.08142v1…