PaperlayerView exact revision on arXiv

Source-linked AI summary

OverForge: Reasoning Through Strategies and Tactics Helps Cooperative Lifelong Adaptation

Oana Madalina Fron, Ojas Shirekar, Chirag Raman

arXiv:2609.39727v1cs.AIcs.CLcs.MA

TL;DR

Cooperative language-model agents need to coordinate over long horizons and adapt to changing tasks and unfamiliar partners, but existing agents choose actions without separating persistent strategies from tactics. OverForge couples strategic and tactical reasoning inside each agent’s partner-conditioned world model, imagining branch outcomes before commitment. In OvercookedV2, the hierarchy improves coordination over flat agents in layouts that afford its proposed strategies, while experiments also examine role retention and adaptation across episodes.

  • Problem

    Cooperative language-model agents that represent teammates still select actions flatly, leaving a predictive mechanism for both strategic and tactical reasoning within each agent unaddressed.

    Source: §1 · p. 1

  • Method

    OverForge uses a training-free hierarchical controller in each agent’s private, partner-conditioned world model to propose strategies and tactics, imagine their consequences, and commit when confident.

    Source: §1 · p. 2, §3 · p. 3, §4.2 · p. 6

  • Results

    OvercookedV2 evidence shows improved coordination over flat agents where layouts afford proposed strategies, retention of agreed roles, and partner-model gains across episodes.

    Source: §1 · p. 2, §5.2 · p. 8, §5.3 · p. 8

  • Takeaways & Limitations

    Preserving coordination agreements can help, but an agreed role is useful only when the kitchen allows the agent to fulfil it; imperfect predictions can still inform choices.

    Source: §6 · p. 9

    Five episodes per condition do not establish robustness across learning histories, models, or human partners.

    Source: §6 · p. 9

Abstract

from arXiv · show

Cooperative language-model agents must coordinate over long horizons and adapt to changing environments and to partners with unfamiliar conventions, yet existing agents map observations to actions without separating persistent coordination strategies from their tactical execution. We introduce OverForge, a training-free hierarchical architecture that separates strategic reasoning over roles and divisions of labour from tactical reasoning over actions within each agent's private, partner-conditioned world model. A metacognitive Prefrontal Cortex Module couples the two levels by forming strategy-action branches, imagining their consequences with a forward model, and committing when confident. In OvercookedV2, OverForge delivers 7 soups in a connected kitchen versus 3 for each flat LLM baseline, retains agreed roles, and adopts roles proposed by unfamiliar partners. Ablations and a fixed-strategy probe show that persistent strategies guide tactical adaptation while each reasoning level contributes to coordination. Memory restarts show that cross-episode partner knowledge supports task performance and partner prediction, linking the hierarchy to continual adaptation.

1 INTRODUCTION

Cooperative agents must sustain goals, track joint state, reason about teammates, and adapt to unfamiliar tasks and partners, while current cooperative language agents still select actions flatly. OverForge addresses this gap with hierarchical reasoning inside each agent’s private, partner-conditioned world model.

1 INTRODUCTION

  • Cooperative agents must sustain goals, track joint state, reason about teammates, and adapt online to unfamiliar tasks and partners.Errors in modelling partners can produce duplicated work, interference, unstable roles, and poor coordination with unfamiliar teammates.Source: §1 · p. 1
  • Existing cooperative language-model agents represent teammates but select actions flatly, leaving a predictive mechanism for both strategic and tactical reasoning within each agent as a gap.World-model methods typically serve a single planner or policy, while joint-embedding predictive architectures address prediction across abstraction levels.Source: §1 · p. 2
  • OverForge uses a training-free hierarchical controller that predicts strategies and tactics inside each agent’s world model and imagines consequences before commitment.Its evidence includes improved coordination over flat agents where the layout affords proposed strategies, role retention and adoption, and ablations of both reasoning levels.Source: §1 · p. 2, Figure 1 · p. 2

2 RELATED WORK

Related work studies hierarchical reasoning, partner modelling and learning across encounters, and planning with predictive world models. OverForge combines these threads by prompting strategic and tactical levels from one frozen model and placing predictive hierarchy within each cooperating agent.

Hierarchical reasoning and human cognition

  • Prior hierarchical and cognitive-agent approaches use trained lower tiers or fixed deliberation depth, whereas OverForge prompts both language levels from one frozen model and lets the upper tier control commitment.Related approaches include prefrontal-style planning, dual-process control, metacognition, and language hierarchies that decompose goals.Source: §2 · p. 2

Theory of mind, cultural learning, and lifelong competence

  • Partner-modelling work addresses unfamiliar partners, while continual-learning work accumulates skills or cultural knowledge; OverForge carries partner models into imagined rollouts and treats strategy as transferable.The related work includes theory-of-mind methods, reflective memory, explicit belief states, and structured social world models.Source: §2 · pp. 2–3

Planning with latent world models

  • OverForge follows JEPA-inspired prediction but searches shallow strategy-and-first-action branches without task reward, placing the hierarchy within each agent rather than a single-agent planner.Other described approaches include language-model reasoning-as-planning with search and learned latent world models.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries distinguish environment-conditioned feasible strategies from state-dependent tactics. They also specify OverForge’s inherited agent components, its two meanings of world model, and controls used to make judge scores more consistent.

Constraints, strategies and tactics

  • Constraints determine which coordination patterns are feasible; strategies persistently organise work, space, and responsibilities, while tactics are state-dependent primitive actions that realise them.The strategy set is conditioned on environmental constraints rather than universal.Source: §3 · p. 3

Base agent and world model

  • OverForge inherits MindForge’s perception, structured belief and theory-of-mind representation, communication, and memory; its addition is the strategic/tactical hierarchy and coupling controller.“World model” refers both to the maintained representation updated across steps and to the generative forward model that imagines predicted states.Source: §3 · p. 3

Reliability of LLM-as-a-judge

  • Evaluation fixes the judge, prompts, scale, criteria, and scoring anchors to improve consistency, while acknowledging residual numerical scoring bias.The resulting scores are used as task-grounded estimates.Source: §3 · p. 3

4 OVERFORGE: REASONING OVER STRATEGIES AND TACTICS

OverForge formalizes cooperation as coordination across environments and partner policies despite agents maintaining distinct private world models. Its hierarchical controller evaluates persistent strategies and executable actions together, using imagined outcomes and confidence to govern commitment.

4.1 PROBLEM STATEMENT

  • Agents build private representations from different observations and cannot inspect a partner’s representation, relying on its behavior and messages.The controller objective is coordinated behavior across rooms, recipes, and partner policies rather than exploiting one kitchen or partner.Source: §4.1 · p. 4
  • Each agent’s controller should coordinate across rooms, recipes, and partner policies rather than exploit one kitchen or partner.Source: §4.1 · p. 4

4.2 HIERARCHICAL PREDICTIVE CONTROL

  • A frozen language model runs a hierarchical control loop inside each agent’s world model, with the PCM coupling strategic and tactical reasoning.The representation is refreshed from events since the previous decision; other agent components are inherited from MindForge.Source: §4.2 · p. 4
  • The PCM pairs persistent, action-free coordination strategies with candidate executable first actions, retaining each strategy to judge whether its action serves its intention.Each strategy-action pair receives its own world-model clone and rollout, so lookahead can affect which action is selected.Source: §4.2 · p. 5
  • Pairwise judgments score immediate actions on coherence, goal-directedness, epistemic value, and social fit, while extra comparisons target uncertain rankings.Every candidate receives at least two comparisons; remaining comparison budget goes to pairs whose accumulated margins are closest.Source: §4.2 · p. 5
  • Partner-conditioned rollouts clone the maintained model, carry teammate models through imagined transitions, and score the deepest imagined kitchen state reached.Episode memory is excluded from the forward model, and the state score rates delivery prospects rather than trajectory return.Source: §4.2 · pp. 5–6
  • The PCM commits to the highest-utility branch when confidence reaches its threshold; otherwise it extends plausible branches until confident or unable to deepen.If confidence never reaches the threshold, it samples a branch from the final posterior rather than resolving ambiguity with an arbitrary tie-break.Source: §4.2 · p. 6

5 EXPERIMENTS

The experiments test OverForge across kitchen layouts, partner pairings, and memory conditions. Results show persistent role retention, contributions from strategic reasoning and rollouts, and cross-episode gains associated with retained partner knowledge.

5.1 GENERAL SETUP

  • Each condition runs five 300-step episodes with randomly sampled recipes; the study uses connected and split kitchens, self-play and cross-play pairings, and several task and coordination measures.All language-model agents share a frozen Qwen-3.5-27B and the same outcome-triggered three-turn dialogue protocol.Source: §5.1 · pp. 6–7

5.2 HIERARCHY VS. FLAT: THE FULL-AGENT COMPARISON

  • OverForge retains agreed roles across steps; in split-room proposals, 81.9% avoid relays and hand-offs, while the remaining 18.1% reflect an unstated environmental constraint.The finisher role is recommitted eleven times in the connected-room example, including a 26-step blockage of the loader.Source: Figure 5 · p. 7, §5.2 · p. 8, Figure 6 · p. 8
  • In the connected room, the full controller delivers 1.4±0.5 soups per episode, versus 0.8±0.8 without rollouts, 0.4±0.5 without the strategic layer, and 1.0±0.9 without pairwise ranking.In the split room, the arm without ranking delivers 0.8±0.7 soups, more than the other ablation arms; removing the strategic layer or rollouts lowers deliveries overall.Source: §5.2 · p. 8, Table 1 · p. 8
  • With a fixed strategy, the tactical level executes the given plan rather than repairing physical infeasibility; only a partner’s concrete offer revises it.Under an all-ingredient_0 order, the pinned strategy names an unreachable location for 158 of 300 steps.Source: §5.2 · p. 8

5.3 LIFELONG ADAPTATION

  • Across episodes, partner-intent accuracy rises from 0.16 to 0.76 for agent_0 and from 0.44 to 0.53 for agent_1, while MindForge’s changes from 0.30 to 0.28.By episode 4, completed sub-tasks per seat rise to 21 and 23; coordination conventions continue to vary across episodes.Source: §5.3 · p. 8
  • With MindForge partners, OverForge counters only 5% and 4% of 101 and 137 role proposals, respectively, but fails 0.80 of its interactions versus 0.29 and 0.14 for MindForge and MindForge-C.It retains 67% of its self-play delivery rate, and the partner serves 5 of the 6 cross-play soups.Source: §5.3 · pp. 8–9, Table 2 · p. 9
  • Across two connected-room memory restarts, teams deliver 2 soups total versus 7 over the corresponding original-run intervals, while the kept-memory seat has higher partner-intent accuracy.The kept-memory advantage is +0.32 and +0.15, compared with original-run seat gaps of +0.01 and -0.23 over the same episode ranges.Source: §5.3 · p. 9

6 DISCUSSION AND CONCLUSION

The discussion connects cooperative performance to preserving role agreements, using imperfect predictions, and distinguishing partner learning from environmental learning. It also identifies limited evaluation breadth and infeasible strategies as boundaries for the findings.

Discussion and conclusion

  • Role agreements can support coordination, but their usefulness depends on whether the kitchen lets agents fulfil them.Removing the strategic layer reduces connected-kitchen deliveries, while split-kitchen failures motivate revising infeasible agreements.Source: §6 · p. 9
  • Removing lookahead rollouts reduces deliveries despite errors in imagined positions, so prediction’s decision effects merit measurement alongside state accuracy.Source: §6 · p. 9
  • Memory erasure disrupts connected-kitchen cooperation, while retained memory does not consistently help in the split kitchen; Figure 7 plots blocked steps and deliveries across restarts.Source: §6 · p. 9, Figure 7 · p. 9
  • Five episodes per condition leave robustness across learning histories, models, and human partners unestablished; further runs are needed.The authors identify learning from unsuccessful attempts as a next step for revising strategies that the environment makes infeasible.Source: §6 · p. 9
  • The study uses simulation only, with no human subjects, personal data, or user study, and evaluates cooperative partners on a toy task.Source: ETHICS STATEMENT · p. 10
  • The paper specifies its environment, agents, conditions, prompts, measures, and complete tables, and says source code, configurations, and run logs will be released with the camera-ready version.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 FULL NOTATION

Table 3 groups notation by decision stage and distinguishes real, inferred, and predicted symbols.

A.1 Full notation

  • The table’s fourth symbol group contains symbols used only in the appendices.Source: §A.1 · p. 16
  • Table 3 organizes symbols by decision stage and distinguishes plain symbols from inferred and predicted ones.Source: Table 3 · p. 16, §A.1 · p. 16
  • The maintained world model is denoted w_i^t, while p_φ denotes the forward model.Source: Table 3 · p. 16

A.2 STRATEGY AND TACTIC EXAMPLES

Table 4 illustrates how layout constraints shape available strategies and their tactical realization.

A.2 Strategy and tactic examples

  • Table 4 lists layout constraints alongside two strategies those constraints afford and tactics that realize each strategy.Source: §A.2 · p. 16
  • Strategies are stated without coordinates or primitive actions, keeping them abstract from their execution.Source: §A.2 · p. 16
  • The same strategy maps to different action sequences as the state changes.Source: §A.2 · p. 16

A.3 REFERENCE-AGENT ADAPTATIONS

The reference agents were adapted to a shared Overcooked-v2 protocol so comparisons could isolate differences associated with the hierarchy. The adaptations also expose distinct limits in what each baseline controls or represents.

A.3 Reference-agent adaptations

  • The adaptations are detailed because attributing agent differences to hierarchy requires holding surrounding components equal.Source: §A.3 · p. 16
  • All language-model agents share a frozen model, observation rendering, three-turn outcome-triggered dialogue, critic, curriculum, skills, episodic memory, and reward handling.Delivery reward is logged but never supplied to prompts; the shared setup creates the MindForge → MindForge-C → OverForge lineage.Source: §A.3 · pp. 16–17
  • MindForge is adapted from an asymmetric Minecraft system to symmetric, in-process Overcooked agents with a partner model and a three-verdict critic.Source: §A.3 · p. 17
  • MindForge-C adds one causal predict-and-reselect pass, but lacks a partner mental model, so its comparison with MindForge does not isolate the forward model alone.Source: §A.3 · p. 17
  • IPPO delivers zero soups in the asymm_advantages evaluation, which is out of distribution in both kitchen layout and padded spatial encoding.The paper cautions that this collapse reflects a policy tied to its training kitchen’s encoding, not reinforcement learning generally.Source: §A.3 · p. 17

A.4 PCM DESIGN DETAILS

The PCM deliberates over strategy–action branches in a private world model, extending imagined rollouts selectively and committing when its branch posterior is confident enough. Only the committed first action reaches the environment; the remaining predictions and branches stay internal.

PCM design

  • The PCM proposes strategies and tactics, ranks each strategy’s candidate actions by pairwise comparison, then initializes and extends the resulting branches.Immediate ranking uses a comparison budget; the accumulated margin steers which pairs receive further comparisons but does not enter the returned action value.Source: §A.4 · pp. 17, 18, 20

Deliberation and commitment

  • Rollouts deepen only branches in the previous round’s smallest high-probability set reaching mass ρ, while confidence is computed across all branches.The depth cap bounds inference; if confidence remains below threshold, the PCM samples an action from the branch posterior instead of committing to the highest-utility branch.Source: §A.4 · pp. 18–19

PCM design

  • Each strategy–first-action pair becomes a separate branch with its own world-model clone, and only the committed first action is executed.Branching at the first action lets lookahead affect which executable action is selected; imagined states do not alter the maintained real-world model.Source: §A.4 · p. 18

Deliberation and commitment

  • Rollout action sampling and branch-posterior formation use separate softmax distributions: one selects deeper actions, while the other sets confidence and branch retention.This keeps incomplete trajectory estimates out of deeper action selection while allowing commitment to use immediate and longer-horizon evidence.Source: §A.4 · p. 18

Logged decision

  • In the logged connected-room decision, four branches favored picking up a plate, and after round 3 the posterior concentrated on b2 at κ = 0.84 ≥ θ.The trace also shows that imagined-state errors can persist: b4 predicts a counter-cell position, while b2 predicts no change at depths 2 and 3 despite the state judge raising ν.Source: Figure 8 · p. 19

Implementation details

  • The fixed configuration uses K = 2 strategies, six comparisons, a five-step rollout cap, θcommit = 0.6, τ = 0.1, and ρ = 0.9.Unparseable self-play comparisons receive neutral 0.5 scores; after commitment, the next real observation updates the maintained model and deliberation restarts.Source: §A.4 · p. 20

A.5 PARTNER CONDITIONS AND MEASURE FAMILIES

This appendix specifies partner conditions and the measures used to assess task performance, coordination, transfer, and PCM behavior. Self-play pairs identical agent types, while cross-play pairs different types without changing the evaluated agent.

Environment and protocol

  • Figure 9 shows the N = 2 starting states for cramped_room and asymm_advantages, with N = 3 starts shown in Figure 18.The evaluated rooms differ in layout and resource arrangement, creating distinct coordination settings.Source: Figure 9 · p. 21

Partner conditions

  • Self-play pairs identical agent types, whereas cross-play pairs different types; the evaluated agent’s configuration stays unchanged between conditions.The partner’s architecture, prompts, beliefs, memory, and communication protocol are identical in self-play, making cross-play the unfamiliar-partner comparison.Source: §A.5 · p. 21

Measure families

  • The evaluation reports task measures, coordination and transfer measures, and PCM trace measures from logged decisions and model calls.The measures include deliveries and curriculum progress; non-progress, blocking, duplicated subtasks, failed interactions, social influence, and self-play-to-cross-play delivery gaps; and rollout, confidence, fidelity, partner-intent, and inference-cost measures.Source: §A.5 · pp. 21–22

A.6 FULL METRIC DEFINITIONS

The appendix defines offline measures from agent-step, PCM-decision, and LLM-call logs to assess task progress, coordination, strategy use, model fidelity, partner prediction, and transfer. It specifies how each rate or score is computed and notes several metric-specific boundaries.

Task measures

  • Deliveries sum team reward divided by 20, while success rate counts episodes with at least one delivery and progress completeness records the furthest soup-pipeline stage reached.A sub-task is completed on a critic success verdict and abandoned after five consecutive failure verdicts; progress completeness grades stages from holding an ingredient through delivery.Source: §A.6 · p. 22

Coordination measures

  • Non-progress combines idle stays and wasted moves per agent-step; blocking, duplicated-sub-task, and failed-interaction rates are separately defined, with lower coordination rates better.IPPO’s missing observation text means pot status is unknown, so all its stays count as idle, affecting at most 5% of its steps.Source: §A.6 · p. 22
  • Realised social influence estimates action-stream mutual information with add-one smoothing and reports the mean positive influence across ordered agent pairs in bits.It can rise with any coupling, including mutual interference, and is computed only when an ordered pair has at least eight samples.Source: §A.6 · pp. 22–23

Partner transfer

  • The partner-transfer gap compares self-play and cross-play deliveries per 1,000 steps, with cross-play pooling runs against different architectures.Every cross-play team includes OverForge, so the other three agent types share the same cross-play value with OverForge.Source: §A.6 · p. 23

PCM trace measures

  • PCM traces report mean rollout depth and utilization, confidence, commit rate, comparison count, branch count, imagined steps, inference cost, near-ties, and within-decision score spread.Inference cost is LLM calls per agent-step; near-ties have top utilities differing by less than 0.02.Source: §A.6 · p. 23

Strategy measures

  • Strategy measures include abstraction ratio, plan stability, role events, plating-to-serving ratio, and tactical divergence between the same seat’s sub-task vocabularies across layouts.The abstraction ratio is a lower bound: in a hand-check, all counted matches aligned, while 13 of 24 non-matches also aligned in meaning.Source: §A.6 · p. 23

World-model and partner measures

  • World-model fidelity scores only the chosen branch against the logged future state, while partner-pose and holding measures check exact teammate predictions and intent accuracy is a token-overlap lower bound.Temporal consistency is 1 at depth 1 by construction; partner-intent accuracy is undefined for IPPO because it exposes no sub-task.Source: §A.6 · p. 23

Dialogue and strategy measures

  • Lexical rules classify sub-task and strategy families, role assignments and uptake, role splits, counter-proposals, joint dishes, unreachable tasks, and tactical detail.Hand-check accuracy varies by rule; counter-proposal rates are approximate because some replies labelled as counters actually accept the proposal.Source: §A.6 · p. 24

A.7 ADDITIONAL RESULTS AND DISCUSSION

OverForge’s free strategic layer adapts roles to the room, while memory restarts and partner-transfer results show partner knowledge persisting across episodes. Further probes locate coordination limits in physical constraints and examine how strategy, tactics, and imagined branches contribute.

A.7 ADDITIONAL RESULTS AND DISCUSSION

  • Imagined states are often wrong about the agent’s position, yet imagined branches can still be ranked usefully because the controller compares options rather than following imagined trajectories.The paper reports a beam of about five branches at mean depth 2 in the connected room.Source: §A.7 · p. 25, Table 9 · p. 25
  • After memory restarts, connected-room partner-intent accuracy reaches 0.84 for the kept seat and falls to 0.26 for the erased seat, while sub-task completions and role events remain within the original-run spread.Restarted-seat partner-model update rates of 0.3–0.7 per episode overlap the original run’s 0.2–0.7 range.Source: Table 10 · p. 27, Figure 11 · p. 27, Figure 12 · p. 28
  • Stalls arise when otherwise appropriate roles meet blocked cells, inaccessible counters, or stations in the other half; persistent strategies can prolong these stalls without causing them.The authors identify missing constraint information and belief updates that weigh partner reports against observed outcomes as relevant remedies.Source: §A.7 · pp. 30, 34
  • 7 soups in the connected room versus 3 for the taxonomy and pinned-strategy arms; the free strategic layer finds room-dependent roles without the taxonomy at roughly half the LLM-call cost.Splitting by item rises from 3.6% to 8.3% without the taxonomy and from 1.0% to 6.0% with it; the table reports 40.2–40.5 versus 80.8–81.8 LLM calls per agent-step.Source: §A.7 · p. 30, Figure 14 · p. 31, Table 11 · p. 33
  • A concrete partner offer revises the pinned strategy once in the connected room, while the pinned agent holds its pot-loader role throughout the split-room episode under an all-ingredient_0 order.The connected-room agent plates at step 132 and serves at 137; in the split room, the pinned seat carries soup toward the serving station for 55 steps.Source: §A.7 · p. 30, Figure 15 · p. 31, Figure 16 · p. 32
  • Without the strategic layer, single-branch decisions yield confidence κ = 1 and commit rates of 0.42 and 0.52, while the full controller commits at 0.12 and 0.14.Without rollouts, confidence concentrates just below the threshold without increasing the commit rate; without pairwise ranking, the commit rate rises to 0.28 in both rooms.Source: Table 6 · p. 24, §A.7 · pp. 30–31, Figure 17 · p. 33
  • A third agent lowers connected-room deliveries from 7 to 2 as blocking rises from 0.22 to 0.53, but raises split-room deliveries from 1 to 3 where independent work can be allocated.In the split room, 9 of 10 dishes are cooked jointly with three agents, versus 2 of 7 with two; duplicated sub-tasks remain low in both layouts.Source: §A.7 · p. 33, Table 12 · p. 34
  • Partner-intent accuracy per predicting seat remains 0.47–0.56 in the connected room and 0.48–0.58 in the split room with two partners to track.Partner-model update rates are 0.57–0.79 with language-model partners and 0.00–0.06 with the silent IPPO partner, for whom the model holds an idle/waiting prediction for 914 of 1,500 split-room steps.Source: §A.7 · pp. 25, 33, 34, Table 8 · p. 25

A.8 BROADER FIELD IMPACT DISCUSSION

OverForge’s strategic layer preserves roles between messages, but its value varies with how many divisions of labour the environment affords. The broader results also point to ranking futures, dialogue-based partner knowledge, and explicit reporting of where inference is spent.

Strategic persistence and environmental affordances

  • 1.4±0.5 vs 0.6±0.8 soups per episode when many divisions are feasible, but 0.2±0.4 vs 0.6±0.8 when one is, making value affordance-dependent.Assigned roles reappear in 0.57–0.77 of proposals versus 0.40–0.69 with shuffled requests; a role was re-committed eleven times, at three times flat-agent inference and with a 26-step stall when held too rigidly.Source: §A.8 · p. 34

Predictive world models and deliberation

  • Removing predictive rank order halves deliveries despite 51% one-step own-position error; replacing pairwise comparison with absolute scores preserves deliveries and sharpens confidence.The metacognitive gate commits at 0.12–0.14 and reaches depth 2.1 of 5, spending more inference where ranking is ambiguous.Source: §A.8 · p. 35

Partner knowledge and continual adaptation

  • Across episodes, partner knowledge improves and transfers after memory restart, while failed-interaction rates remain 0.65–0.78 across architectural changes.The partner model updates in 0.57–0.79 of steps with a talking partner versus 0.00–0.06 with a silent one; the discussion calls for grounding and causal attribution of failed actions to constraints.Source: §A.8 · p. 35

A.9 CONVERSATION EXAMPLES

The conversation examples show OverForge committing to roles, adapting them to partner messages, and carrying partner beliefs across memory restarts. They also illustrate cases where a pinned strategy or an unstored constraint limits adaptation.

First conversation

  • At step 3, OverForge’s agent commits to fetching and staging ingredients at κ = 0.78 after sampling at κ = 0.23; Figure 19 contrasts its role commitment with MindForge’s next-move choice.The figure depicts strategies, first actions with utility U, the chosen strategy, and confidence κ before and after the PCM decision.Source: §A.9 · p. 35, Figure 19 · p. 36

Role split and repair

  • In episode 5, the finisher role remains held from plate pickup at step 194 through the serve at step 293, despite a 26-step blockage.The step-226 exchange changes the agent’s position, not its role.Source: §A.9 · p. 36

Pinned strategy probe

  • The pinned seat gives up g0 once in the connected-room probe, plating at step 132 and serving at step 137.This was its only plating and serve in that room.Source: §A.9 · p. 36
  • Under the split-room all-ingredient_0 order, 158 of 300 steps name locations the pinned seat cannot reach, limiting revision of g0.The ingredient pile and serving station are both unreachable from the pinned seat’s half of the room.Source: §A.9 · pp. 36–37

Partner interaction

  • When MindForge proposes a hand-off plan, OverForge adopts it and adds timing, while countering 4% of the 137 proposals it receives.The example shows adoption of the partner’s plan rather than frequent counter-proposals.Source: §A.9 · p. 37
  • After a partner corrects OverForge’s belief about who holds an ingredient, the correction enters its interaction beliefs and informs its next strategic call.The partner model update rate with MindForge partners is 0.57–0.58.Source: §A.9 · p. 37

Memory restart

  • After memory restart, a kept seat opens with a prior partner belief or division of labour; the fresh seat corrects the first and adopts the second.The examples show the kept seat carrying information across the cut into a new episode.Source: §A.9 · p. 37

Flat-agent examples

  • Flat controls can settle only the next move at κ = 1, while a named constraint may remain unstored and be repeated even when its referenced tool is absent.The constraint example has the same seat announce fetching ingredient_0 again one step later, although the room has no cutting board.Source: §A.9 · p. 37

A.10 PROMPTS AND RESPONSE TEMPLATES

This section documents prompt templates and response formats for agent action selection, belief updates, partner-aware communication and modeling, prediction, critique, and subtask selection. It also specifies prompts for separating strategic plans from tactical actions and supporting curriculum generation, memory, skill reuse, and fixed-strategy probe variants.

Base system and environment

  • The base prompt requires one action per step in strict JSON while prioritizing immediate subtasks and completing as many recipes as possible during the episode.Agents receive task and environment context, past episodes, skills, beliefs, teammate communication, and an interaction preview to guide action selection.Source: §A.10 · p. 38

MindForge prompts

  • MindForge acts once per step and communicates in separate turns after subtask success or failure, reasoning from distilled beliefs and mental models rather than raw chat logs.Its action prompt includes task, observation, skill and episode summaries, beliefs, and world-model context.Source: §A.10 · p. 40

Belief system (four-facet belief)

  • For two-agent runs, one prompt updates perception, partner, interaction, and task beliefs together; separate per-facet prompts are reserved for teams of three or more.The combined prompt asks for concise factual beliefs about surroundings, teammates’ likely states and intentions, coordination-relevant messages, and subtask progress or blockers.Source: §A.10 · pp. 40–41

Belief prompts

  • Belief prompts extract up to five directly useful teammate beliefs and merge task context into two to four sentences about progress, the next step, and blockers.Beliefs cover coordination agreements, observed kitchen state, warnings, requests, and handoffs; both prompts require JSON output.Source: §A.10 · p. 42

Theory of mind and communication

  • Communication prompts use partner beliefs, perspective, current action, and kitchen information to produce concise, natural, actionable messages, while avoiding generic or assumed knowledge.Messages should explain what the agent is doing and why, and can convey ingredient locations, pot states, next steps, warnings, or handoffs.Source: §A.10 · pp. 42–43
  • For coordination conversations, agents use structured mental models and teammate perspective to propose specific work division, locations, warnings, or handoffs in one concise message.A separate perspective analysis asks what teammates likely understand, are doing, and need, grounded in the conversation and mental model.Source: §A.10 · pp. 43–44
  • Agents update teammate models from observed position, held items, and messages, inferring partners’ tasks, intentions, needs, coordination agreements, and beliefs while marking unknowns.The prompt says to update only inferred fields and stay grounded in what was observed or actually said.Source: §A.10 · p. 45

A.10 PROMPTS AND RESPONSE TEMPLATES

  • The forward-model prompt treats an action as a causal intervention, predicts downstream effects, preserves invariant state, and returns the complete next state in a fixed JSON schema.A companion prompt adds task beliefs and a structured teammate model to anticipate likely concurrent moves and their effect on the joint outcome.Source: §A.10 · pp. 46–47
  • The critic compares before-and-after states to classify the last action as success, progress, or failure based on the subtask’s concrete goal condition.It treats a no-op interaction as failure, movement toward a target as progress, and requires an actual world change—not position or facing alone—for success.Source: §A.10 · pp. 47–48
  • The curriculum prompt selects one concrete immediate subtask from the current state and recipe, moving on after success or retaining the goal with a different approach after failure.It encourages repeated subtasks as needed and choosing work that complements rather than duplicates what the teammate is handling.Source: §A.10 · p. 49
  • Curriculum prompts generate 5–8 self-contained questions about the cooking pipeline, and skill prompts describe successful actions or query a library for useful past skills.The curriculum questions focus on the soup-delivery goal; skill descriptions express functional outcomes rather than repeating action names.Source: §A.10 · pp. 49–50
  • Episodic-memory prompt asks for a concise summary of past sub-task attempts, highlighting successes and how, failures and why, plus useful patterns or lessons; semantic prompt seeks generalizable domain facts, not episode-specific observations.Each fact is on a separate line and is one sentence under 20 words.Source: §A.10 · p. 51
  • Strategic prompts propose high-level roles and divisions of labour, while tactical prompts generate concrete action candidates for a plan or directly in the no-hierarchy arm.The prompts condition planning and action selection on the world model and relevant memory.Source: §A.10 · pp. 51–52
  • The pairwise judge scores actions equally across coherence, goal-directedness, epistemic value, and social fit, then uses the score gap as confidence.It compares how well each action advances shared work and partnership, rather than maximizing a reward.Source: §A.10 · pp. 52–53
  • The absolute judge applies the same four-facet average to one action alone, without reporting a better action or confidence.It instructs the model to score the action on its own merits and avoid inventing objects or positions.Source: §A.10 · p. 53

Judge response templates

  • The judge templates constrain action scores to four 0–1 facets with an equal-weight mean and constrain state evaluation to a JSON value rating team progress toward delivering soups.The action response requires coherence, goal_directedness, epistemic_value, social_fit, score, and a reasoning sentence; the state judge rates promise on a 0.0–1.0 scale.Source: §A.10 · pp. 53–55

Fixed-strategy probe prompts

  • The fixed-strategy probe replaces both seats’ layer prompts with taxonomy prompts in one arm, while the pinned arm bypasses one seat’s strategic call and fixes its plan as pot-loader.The pinned strategy is to keep a pot filled with ingredients needed for the order while leaving plating and serving to the teammate.Source: §A.10 · pp. 55–56

Strategic taxonomy prompt

  • The strategy prompt defines strategies as standing roles or divisions of labor, distinct from short-lived tactics, and offers six role families while allowing alternatives that fit the state and teammate.The families are supplier, cook, finisher, split-by-item, relay, and yielder; strategies must not name coordinates or moves.Source: §A.10 · p. 55

Tactical taxonomy prompt

  • The tactical prompt selects concrete next moves to serve the standing strategy, adapting to the current state and teammate while restricting outputs to six action primitives.It supplies recurring patterns including approach-and-face, re-face, stage, yield, hold, and check-the-order.Source: §A.10 · pp. 55–56
Loading 2609.39727v1…