PaperlayerView exact revision on arXiv

Source-linked AI summary

OpenSocialJax: Evaluating Social Adaptation in Sequential Social Dilemmas with Open-Ended Environmental Dynamics

Zihao Guo, Meng Fang, Joel Z. Leibo, Yali Du

arXiv:2610.13710v1cs.MA

TL;DR

Sequential social-dilemma benchmarks rarely test agents’ joint adaptation to hidden environmental rules and other agents. OpenSocialJax varies latent layouts and mechanisms across repeated trials and evaluates eight language models. Rule discovery often improves performance but does not reliably sustain cooperation across the two environments.

  • Problem

    Benchmarks rarely test whether agents can infer hidden environmental rules through repeated interaction while also adapting to other agents in sequential social dilemmas.

    Source: §1 · pp. 1–2

  • Method

    OpenSocialJax procedurally varies layouts and hidden rules across episodes, holds each configuration fixed across trials, and evaluates eight language models in homogeneous and mixed teams.

    Source: §1 · p. 2

  • Results

    Rule discovery often improves performance, but cleaning becomes concentrated among fewer agents in OpenCleanup and every team eventually depletes the shared resource in OpenHarvest.

    Source: §5 · p. 9

  • Takeaways & Limitations

    The results show that adapting to unfamiliar environmental dynamics does not reliably produce sustained cooperation in these social dilemmas.

    Source: §5 · p. 9

    Training learning-based agents that adapt to the latent configuration remains an open challenge.

    Source: §5 · p. 9

Abstract

from arXiv · show

Sequential social dilemma benchmarks capture diverse social interactions, but rarely test whether agents can infer hidden environmental rules through repeated interaction while adapting to other agents. We introduce \textbf{OpenSocialJax}, which extends Clean Up and Commons Harvest across an open-ended distribution of latent environment configurations, systematically varying spatial layouts and hidden rules governing tool crafting, payoffs, and resource dynamics. Each episode samples one configuration from tens of millions of possibilities and holds it fixed across five trials. We evaluate eight large language models, including three closed source models and five open source models. Our evaluation tests whether models can discover hidden rules and adapt to other agents in homogeneous and mixed teams. We find that rule discovery often improves performance but does not reliably sustain cooperation, as the cleaning burden becomes concentrated among fewer agents in OpenCleanup and every team eventually exhausts the shared resource in OpenHarvest. In our mixed arena, one cooperative agent was able to sustain the public good in OpenCleanup at a cost to its own return, but could not prevent resource depletion in OpenHarvest. Together, these results highlight the difficulty of adapting jointly to unfamiliar environmental dynamics and the changing behavior of other agents.

1 INTRODUCTION

Existing benchmarks rarely test whether agents can infer hidden environmental rules while adapting to other agents, leaving their joint adaptation in social dilemmas unclear. OpenSocialJax addresses this gap with procedurally varied environments and repeated-trial evaluations of language models.

1 INTRODUCTION

  • Existing social-dilemma benchmarks do not vary hidden environmental dynamics, and LLMs’ ability to adapt when rules and social dependencies vary jointly remains unclear.The paper frames adaptation as requiring inference about both how the environment works and how other agents behave.Source: §1 · pp. 1–2
  • OpenSocialJax varies layouts and hidden mechanisms across episodes while keeping each sampled configuration fixed across multiple trials, enabling agents to learn through interaction.Resampling creates a broad distribution of unfamiliar conditions rather than a fixed set of scenarios.Source: §1 · p. 2
  • The evaluation covers eight recent open- and closed-source models and examines them in homogeneous populations and mixed arenas.The contributions also describe a repeated-trial protocol in which agents retain notes to preserve discoveries across trials.Source: §1 · p. 2
  • Even the strongest model can reason explicitly about the dilemma yet still deplete the shared resource or be exploited by other agents.The paper analyzes exploration, exploitation of discovered rules, and free riding.Source: §1 · p. 2

2 RELATED WORK

Prior work provides open-ended task spaces, sequential social-dilemma environments, and LLM agents for interactive games. The paper distinguishes OpenSocialJax by combining latent environmental variation with social adaptation.

2 RELATED WORK

  • Open-ended benchmarks span generated task spaces, robotics, latent causal structures, survival, physics-based control, and multi-agent adaptation.Examples include XLand, XLand-MiniGrid, AdA, MAESTRO, and Multi-Agent Craftax.Source: §2 · p. 2
  • Sequential social-dilemma benchmarks include Melting Pot, SocialJax, and JaxMARL environments, but generally use predefined settings rather than varying hidden mechanisms.OpenSocialJax instead varies hidden environmental mechanisms to study adaptation across latent social tasks.Source: §2 · p. 3
  • LLM game research studies interactive reasoning and multi-agent behavior, but how agents infer latent rules while adapting to changing environmental mechanisms and inter-agent dependencies remains underexplored.Prior examples include shared-resource depletion in GovSim and LLM-augmented agents in MeltingPot.Source: §2 · p. 3

3 OPENSOCIALJAX

OpenSocialJax evaluates whether agents can adapt to unfamiliar environmental rules and other agents’ behavior in procedurally varied sequential social dilemmas. Its environments preserve a sampled configuration across repeated trials, allowing evidence and strategy to carry over.

3 OPENSOCIALJAX

  • OpenSocialJax varies environment layouts and hidden rules so LLM agents must infer unfamiliar mechanisms and respond to other agents’ observed behavior.The benchmark extends sequential social dilemmas to a distribution of latent conditions.Source: §3 · p. 3

3.1 PROBLEM FORMULATION

  • A latent configuration can alter the state space, transitions, observations, rewards, and initial-state distribution of the partially observable Markov game.The configuration remains unobserved while agents infer it through interaction and adapt to both the environment and other agents.Source: §3.1 · p. 4
  • Each episode samples one configuration and resets the environment between trials without changing it, while agents retain summarized interaction history.This lets agents refine their understanding of the same configuration across trials.Source: §3.1 · p. 4

3.2 ENVIRONMENT DESIGN

  • OpenCleanup makes apple rewards depend on river cleanliness and hides a tool system with 57,657,600 possible rule sets.OpenHarvest varies patch colors, layouts, rewards, and regrowth rules, yielding 60,466,176 possible hidden rule sets.Source: §3.2 · pp. 4–5

3.3 LLM AGENT INTERACTION PROTOCOL

  • Agents cannot see hidden rules, other agents’ prompts or scores, or communicate; objective experience records and free-form notes carry evidence and plans across five trials.At each step, agents receive their own observations, action effects, rule evidence, recent steps, observations of others, earlier-trial summaries, and notes.Source: §3.3 · pp. 5–6

4 EXPERIMENTS

Across repeated trials, models learn hidden rules and sometimes improve returns, but social dilemmas persist: cleaning concentrates among fewer agents, and shared orchards are exhausted. In mixed teams, one agent can sustain the public good at a cost to itself, but cannot prevent resource depletion.

4.1 EXPERIMENTAL SETUP

  • The evaluation uses eight models in homogeneous teams across three seeds per environment, with each seed sampling one configuration for five trials of up to 400 steps.The mixed arena includes Claude Opus 5.5, GPT-6-Sol, GLM-5.3-Flash, DeepSeek-V4.1-Flash, and Qwen3.8-27B.Source: §4.1 · p. 6

4.2 MAIN RESULTS

  • Claude Opus 5.5 is strongest in both environments; in OpenHarvest, all models beat the scripted greedy policy, but only Claude Opus 5.5 and GPT-6-Sol clearly separate from it.In OpenCleanup, Claude’s return rises after the first trial and remains elevated, while several models’ gains peak in trial two and then decline.Source: §4.2 · p. 6, Figure 3 · p. 7
  • OpenCleanup agents learn nearly all recipes they will discover within two trials, while knowledge of OpenHarvest color–ripeness payoffs grows across all five trials.No OpenCleanup agent learns every recipe because it only learns from its own shots; agents can specialize in different tools and divide the cleaning work.Source: §4.2 · p. 6, Figure 4 · p. 7

4.3 SOCIAL DILEMMA ANALYSIS

  • In OpenCleanup, cleaning concentrates among fewer agents across trials; Claude Opus 5.5 maintains the broadest participation, while Qwen3.5-4B’s increase reflects delayed recipe mastery.Agents may infer that others’ cleaning will keep the river productive, then focus on harvesting; because they observe only their own apple rewards, they may overlook the team’s contribution.Source: §4.3 · p. 7, Figure 5 · p. 8
  • No OpenHarvest team keeps the orchard alive for a full trial; Claude Opus 5.5 delays depletion longest, but its orchard lifetime peaks in trial two before shortening.From trial three onward, almost all Claude agents conclude restraint does not pay when others harvest the patches.Source: §4.3 · p. 8
  • Claude Opus 5.5 combines the highest return with one of the most even return distributions in both environments, while low-return models tend to be more unequal.In OpenHarvest, all models except Claude Opus 5.5 cluster near the greedy baseline with similar inequality.Source: §4.3 · p. 8

4.4 OPENSOCIALJAX ARENA

  • In the mixed OpenCleanup arena, Claude Opus 5.5 bears most of the cleaning cost, sustains the public good, and earns less than in its homogeneous team.The mixed team collects more apples than every homogeneous team except Claude’s; models contributing least to cleaning gain the most.Source: §4.4 · p. 9, Figure 6 · p. 9
  • In the mixed OpenHarvest arena, Claude Opus 5.5 loses its homogeneous-team advantage, and the orchard depletes as quickly as in other models’ homogeneous teams.Claude soon stops leaving apples to regrow because other agents harvest the patches; the mixed team depletes the orchard far earlier than the homogeneous Claude team.Source: §4.4 · p. 9, Figure 6 · p. 9

5 CONCLUSION

OpenSocialJax evaluates adaptation to hidden environmental dynamics and other agents in sequential social dilemmas, finding that rule discovery does not reliably sustain cooperation. Training learning-based agents to adapt to latent configurations remains an open challenge.

5 CONCLUSION

  • Across eight language models, rule discovery often improved performance but did not reliably sustain cooperation: cleaning concentrated among fewer agents, and every OpenHarvest team depleted the shared resource.The evaluation covered homogeneous teams and a mixed model arena.Source: §5 · p. 9
  • OpenSocialJax extends Clean Up and Commons Harvest with procedurally generated layouts and hidden rules governing tool effects, rewards, and resource dynamics across tens of millions of configurations.The benchmark evaluates homogeneous teams and a mixed model arena.Source: §5 · p. 9
  • Training learning-based agents to adapt to latent configurations remains an open challenge; future work could add explicit communication and more sophisticated theory-of-mind mechanisms.Source: §5 · p. 9

AI USE STATEMENT

This section describes how LLMs served as evaluated agents and assisted with manuscript work, and illustrates their observation-based action interface, hidden-rule hypotheses, and use of accumulated notes to guide later decisions.

AI USE STATEMENT

  • The authors used GPT and Claude for drafting, editing, and related-work searches and Claude Code for code assistance, but not to generate or modify results or agent transcripts.All AI-assisted content was reviewed, checked against the data and code, and revised by the authors.Source: AI USE STATEMENT · p. 10

A LLM AGENT

  • Each agent is an independent call to its model, receives an environment prompt and its own state message, and returns one JSON action without agent-to-agent messaging.Agents can observe others’ actions only when those actions are in their observation space.Source: A LLM AGENT · p. 13
  • Agents receive a structured local observation that includes learned rules, notes, earlier trials, other agents in view, recent events, the current view, and available actions.OpenHarvest observations also include patches seen so far and emptied cells the agent is waiting on.Source: A LLM AGENT · pp. 13, 16
  • Across trials, the harness preserves discovered rules and trial summaries, while agents edit their own notes and carry forward only a short lessons-and-plans rewrite.Step-by-step notes are dropped at trial end; if the rewrite fails, the agent starts the next trial without notes.Source: A LLM AGENT · p. 13
  • In another excerpt, the agent’s notes advise checking river progress when apples stop spawning, prioritizing visible apples, and keeping tool [0,3] for waste.The notes are updated across later steps of the trial.Source: A LLM AGENT · p. 17
  • The OpenHarvest prompt reports the agent’s trial and step, payoff rules, regrowth observations, ripening times, and whether any apples rotted.The example is from agent B at step 40 of trial 1.Source: A LLM AGENT · pp. 18–19
  • The agent records that ripe cyan apples pay less than unripe ones and that hanging apples support regrowth, shaping its harvesting strategy.Its note advises avoiding clearing the patch while testing over-ripe fruit.Source: A LLM AGENT · p. 19
  • The prompt summarizes other agents’ observed locations and actions alongside the agent’s current goal and earlier completed or abandoned goals.The example’s goal is to test the payoff of green over-ripe apples.Source: A LLM AGENT · p. 19
  • The agent’s view includes patch and map information, while the available actions are listed explicitly and the response is required to be JSON.The excerpt shows this structure for the OpenHarvest prompt and its current state at step 40.Source: A LLM AGENT · pp. 19–20

B MODELS AND INFERENCE

The model settings are shared across both environments, with specific inference choices for Claude Opus 5.5 and DeepSeek V4.1 Flash. The experiments use five-agent teams and trials capped at 400 steps.

B MODELS AND INFERENCE

  • Claude Opus 5.5 replies are parsed from free text because it declined structured output, while DeepSeek V4.1 Flash runs without thinking due to overthinking and excessively long traces.Source: B MODELS AND INFERENCE · p. 20
  • Qwen3.5-4B uses top-p 0.95, top-k 20, presence penalty 1.5, and a 3000-token cap on thinking.Source: Table 1 · p. 20
  • Table 2 summarizes experiment models, environments, seeds, and trials; all experiments use five-agent teams, and trials last at most 400 steps.OpenHarvest trials end early when no apple remains; each seed samples one latent configuration, and the same seeds are shared across experiments.Source: Table 2 · p. 20, B MODELS AND INFERENCE · p. 20

C.1 SCHELLING DIAGRAMS

The Schelling diagrams show that both environments meet sequential social-dilemma conditions, but their cooperation incentives differ. OpenHarvest rewards defection across co-player mixes, while OpenCleanup lacks the fear condition because a lone cooperator can outperform mutual defection.

C.1 SCHELLING DIAGRAMS

  • GPT-6-Luna agents received one of two roles, with otherwise identical prompts and individually paid returns.The defector prompt encouraged self-interest but did not forbid cleaning, and agents were not told others’ roles.Source: §C.1 · p. 21
  • The experiment varied the number of cooperators from zero to five, rotating their identities across three seeds and averaging returns over five trials.Each seed was one episode of five 400-step trials; roles stayed fixed and hidden rules were not revealed.Source: §C.1 · p. 21
  • OpenHarvest satisfies both fear and greed: defectors earn more at every level of cooperator participation, while OpenCleanup satisfies greed but not fear.In OpenCleanup, exploiting four cooperators pays more than mutual cooperation, but a lone cooperator among defectors earns more than a defector under mutual defection because apples grow only if someone cleans.Source: §C.1 · p. 21
  • The diagrams compare mean returns for a cooperator and defector against the number of their cooperative co-players.At a fixed co-player count, comparing the curves shows whether cooperation or defection pays more.Source: §C.1 · p. 21

C.2 TOTAL CLEANING IN OPENCLEANUP

OpenCleanup cleaning trajectories distinguish persistent high effort from declining team cleaning across models. The comparison uses team waste cleared in trials 2 and 5, after agents have had a trial to learn tool recipes.

C.2 TOTAL CLEANING IN OPENCLEANUP

  • Claude Opus 5.5 and Gemma 4 31B maintain high cleaning, consistent with specialization, while GPT-6-Sol, GPT-6-Luna, Qwen3.8-27B, and DeepSeek-V4.1-Flash clean less later, consistent with free-riding.GLM-5.3-Flash declines slightly but its returns fall; Qwen3.5-4B cleans more over time as it learns recipes.Source: §C.2 · p. 21, Figure 8 · p. 22
  • Figure 8 measures waste tiles cleared per agent as team total divided by five, with shaded bands showing ±1 s.d. across three seeds.The comparison focuses on trials 2 and 5 because agents are still learning tool recipes in trial 1.Source: §C.2 · p. 21, Figure 8 · p. 22

C.3 WHY OPENCLEANUP RETURNS JUMP AFTER THE FIRST TRIAL

After the first trial, OpenCleanup agents carry learned recipes and layout notes into later trials, reducing exploration and speeding up cleaning and apple production.

C.3 WHY OPENCLEANUP RETURNS JUMP AFTER THE FIRST TRIAL

  • From trial 2 onward, most models clear their first cell within 10–25 steps, versus 60–70 steps for most teams in trial 1.Trial 1 uses a quarter to a third of steps testing pick-up orders; later, exploration almost stops, first-50-step cleaning rises several-fold, and apples appear much earlier. Claude Opus 5.5 clears its first cell after about 20 steps in trial 1, then its teams do nearly twice as much early cleaning as the next model and produce first apples within about 50 steps.Source: §C.3 · p. 22

C.4 TOOL USE

Knowing more tool recipes is associated with more pickups and switches, yet greater cleaning efficiency. Some models also lose efficiency by recrafting tools they have just lost.

C.4 TOOL USE

  • Agents knowing two or three recipes clean about 15.6 cells per 100 crafting-or-cleaning steps, versus 9.3 for agents knowing one recipe.Within runs and trials, recipe count correlates with pick-ups (median Spearman ρ = 0.71 over 108 run–trials) and tool switches (ρ = 0.82). Multiple recipes let an agent complete the dark-to-mid-to-light-to-water sequence itself, which requires tool switching.Source: §C.4 · p. 22
  • DeepSeek V4.1 Flash and Qwen3.5-4B often recraft a working tool after an extra pick-up, spending up to about twice and about five times as many crafting steps per cleaned cell, respectively.Most models need close to the minimum of two pick-ups per working tool; Claude Opus 5.5 needs fewer because the sliding window can let one extra pick-up complete the next recipe.Source: §C.4 · pp. 22–23
  • Table 4 covers trials 2–5 and reports pick-ups, pick-ups per working tool, recrafting after tool loss, and crafting steps per cell cleared or lightened.Two pick-ups per working tool is the stated minimum unless a sliding window completes a recipe with one pick-up.Source: §C.4 · p. 22, Table 4 · p. 23

C.5 KNOWING THE PAYOFFS VERSUS USING THEM

In OpenHarvest, similar payoff knowledge by trial 5 does not translate into similar returns; how agents time apple consumption differs. Table 6 presents agents’ own end-of-trial lessons.

C.5 KNOWING THE PAYOFFS VERSUS USING THEM

  • Claude Opus 5.5 earns more points per apple and keeps its orchard alive longer by using the best-paying stage more often.Source: §C.5 · p. 23
  • By trial 5, most models know a similar share of OpenHarvest payoffs, but Claude Opus 5.5 takes almost three quarters of apples at their best-paying stage.Other models take apples at that stage for a third to a half, and eat 38–55% before it.Source: §C.5 · p. 23, Table 5 · p. 23, Figure 9 · p. 24

D AGENTS’ OWN NOTES

Loading 2610.13710v1…