PaperlayerView exact revision on arXiv

Source-linked AI summary

Recursive Reasoning or Statistical Extrapolation? In-Context Learning in Multi-Agent Interdependent Decision-Making

Yu Liu, Wenwen Li, Yifan Dou, Guangnan Ye

arXiv:2609.18591v1cs.AIecon.GNq-fin.EC

TL;DR

The paper asks whether ICL improvements in LLM decision-making reflect strategic reasoning or extrapolation of statistical patterns, a question difficult to distinguish in single-agent or cooperative settings. It tests the alternatives in interdependent public goods games against a history-independent REE benchmark, finding that context benefits depend on feedback structure and weaken as interdependence increases.

  • Problem

    It is unclear whether ICL-driven behavioral improvement reflects strategic correction of decisions or extrapolation of context patterns, and the two mechanisms are hard to distinguish in single-agent or purely cooperative settings.

    Source: §1 · p. 1

  • Method

    The study varies interdependence and historical feedback structure in multi-agent games, comparing behavior with a history-independent REE benchmark.

    Source: §1 · p. 1, §2.2 · pp. 2–3

  • Results

    Monotonic trends let history substantially reduce ED and bring the regression slope close to −10, while trendless sequences leave ED high and β = 0.75 yields worse decision quality than β = 0.25 across trajectories.

    Source: §4.4 · p. 7

  • Takeaways & Limitations

    In these multi-agent interdependent scenarios, ICL behavior is more consistent with statistical extrapolation than recursive belief updating toward equilibrium play.

    Source: §6 · p. 9

    The study evaluates only two model families and one public goods game, so its observed patterns may not generalize across architectures or game structures.

    Source: §7 · p. 9

Abstract

from arXiv · show

In-context learning (ICL) enables large language model (LLM) agents to improve decisions using interaction history, yet it remains unclear whether such improvement reflects refined internal reasoning or mere extrapolation of statistical patterns. To disentangle these mechanisms, we study LLM agents in multi-agent incomplete-information games that require recursive belief reasoning. By constructing a public goods game and manipulating the statistical structure of historical feedback, we evaluate decision quality against a history-independent rational expectations equilibrium (REE) benchmark. Our experiments reveal that when historical statistical patterns are disrupted, the benefits of longer context largely vanish, degrading decision quality to the no-context baseline in a way sharply amplified by stronger strategic interdependence. These results suggest that, in such strategic environments, ICL behavior is more consistent with statistical extrapolation than with strategic reasoning. Our work extends the mechanistic study of ICL to strategic multi-agent settings, introduces REE as a diagnostic tool for distinguishing reasoning from extrapolation, and provides a reusable framework for probing the boundaries of LLM reasoning in recursive belief tasks.

1 Introduction

The paper asks whether ICL improvements reflect strategic reasoning or statistical extrapolation, a distinction that is hard to make in single-agent or cooperative settings. It tests this question in an interdependent public goods game and finds that benefits of context depend on historical feedback patterns and weaken as interdependence rises.

1 Introduction

  • The study uses multi-agent games to distinguish reasoning from extrapolation because actions depend on expectations about others.In single-agent or purely cooperative settings, the two mechanisms are often observationally equivalent.Source: §1 · p. 1
  • The experiment uses a repeated n-person public goods game and manipulates interdependence strength and the statistical structure of environmental feedback.Behavior is evaluated against REE policies, which depend on strategic fundamentals rather than historical realizations.Source: §1 · pp. 1–2
  • Under monotonic feedback, ICL improves over baseline, but under jump sequences longer-context gains largely disappear and higher β further reduces the residual gain.The authors interpret this pattern as more consistent with statistical extrapolation than recursive belief revision toward equilibrium play.Source: §1 · p. 2
  • The paper extends mechanistic study of ICL to multi-agent interdependent games that require recursive belief reasoning.It aims to probe ICL limits and identify boundary conditions for recursive belief reasoning in LLMs.Source: §1 · p. 2
  • The paper introduces history-independent REE as a benchmark for distinguishing reasoning from extrapolation in LLM agents.It also proposes a reusable framework that separates interdependence strength from feedback statistics for studying recursive belief tasks.Source: §1 · p. 2

2 Problem Formulation

The paper formulates an incomplete-information public goods game, derives a history-independent rational expectations benchmark, and contrasts recursive reasoning with statistical extrapolation. The resulting predictions link sensitivity to price-sequence structure with strategic interdependence.

2.1 Game-theoretic Model

  • Each of n agents has a private value and chooses whether to participate after observing a public cost; total participation is N_t.The public project generates positive externalities.Source: §2.1 · p. 2
  • As β grows, payoffs become more sensitive to beliefs about others' actions, increasing the role of higher-order expectations.At β = 0, the game reduces to an independent decision-making problem.Source: §2.1 · p. 2

2.2 Rational Expectations Equilibrium

  • REE assumes a shared threshold strategy: agents participate when their private value reaches the cutoff, and beliefs about others match actual play.The paper uses proximity to REE as a behavioral proxy, premised on closer alignment reflecting deeper iterative reasoning.Source: §2.2 · p. 2
  • For a marginal agent, expected participation by the other n − 1 agents is (n − 1)(1 − F(θ*)) under the threshold strategy.The marginal agent's expected total participation includes their own participation.Source: §2.2 · pp. 2–3
  • The REE cutoff is determined by a fixed-point equation and has a unique solution under suitable regularity conditions on F.The passage states that the proof of existence and uniqueness is in Appendix A.Source: §2.2 · p. 3
  • The REE threshold depends on current public cost, β, and F, but not on the historical price sequence.Thus, for the same current cost, the threshold is unchanged across stationary fluctuations, monotonic trends, or irregular jumps.Source: §2.2 · p. 3

2.3 Research Hypotheses

  • The paper contrasts recursive reasoning, which follows the REE reaction function, with extrapolation from historical (p, N) patterns.The two hypotheses predict differing sensitivity to the statistical structure of the sequence.Source: §2.3 · p. 3
  • If reasoning dominates, reordering the same prices should not systematically change decision quality; if extrapolation dominates, trends should outperform irregular sequences, especially at high β.The predicted gap under extrapolation is attributed to amplification through strategic complementarity.Source: §2.3 · p. 3

3 Experimental Design

The experimental design varies strategic interdependence and the temporal structure of public costs while presenting agents with interaction history. It measures collective behavior against a normalized REE benchmark using deviation, fit, and regression measures.

3.1 Game Environment and Control Logic

  • Each session has six rounds in a 50-agent public goods game, with β set to 0.25 or 0.75 and private values drawn from U[0, 49].Agents observe public cost and prior interaction history before independently deciding whether to participate; the environment then aggregates participation and returns payoffs.Source: §3.1 · p. 3

3.2 Price Sequence Structure

  • The design fixes the price set and varies only its order, creating monotonic sequences with a clear trend and jump sequences without a stable trend.This tests whether decisions at the same current price vary with sequence structure, as predicted by extrapolation but not recursive reasoning.Source: §3.2 · pp. 3–4

3.3 Context Structure

  • Agents receive public signals, total participation, and individual payoffs from the most recent k rounds as history.The experiment manipulates context length using c ∈ {1, 7, 13} messages.Source: §3.3 · p. 4

3.4 Evaluation Metrics and Models

  • The evaluation normalizes public cost to [0, 5], where the REE becomes a β-independent linear benchmark, and reports results for GPT-5 and Qwen3-Plus.The main text presents representative GPT-5 results; cross-model comparisons are in Appendix F.Source: §3.4 · p. 4, Equilibrium Coefficient of Determination · p. 4

Equilibrium Deviation

Equilibrium Coefficient of Determination

  • The equilibrium coefficient of determination captures how well the equilibrium line explains participation fluctuations, while strategic regression estimates the response rule from price-participation observations.The regression uses ordinary least squares on per-round observations (p̃_t, N_t).Source: Equilibrium Coefficient of Determination · p. 4

4 Experimental Results and Analysis

Across price trajectories, ICL improves decision quality when feedback has an extrapolable trend, but those gains largely disappear when the trend is disrupted; stronger interdependence worsens performance. The results also show systematic deviations from REE even without interaction history.

4.1 Static Baseline

  • Without history, participation responds too weakly to price, and stronger interdependence nearly doubles ED while cutting R²eq by more than half.The response overestimates participation at high prices and underestimates it at low prices.Source: §4.1 · p. 5, Figure 3 · p. 5

4.2 Monotonic Sequences

  • For β = 0.25, ED drops from 6.18 to 0.84 with three history rounds; for β = 0.75, it drops from 11.14 to 5.60, but bias remains.Gains show diminishing returns from k = 3 to k = 6, and regression slopes approach −10 only at β = 0.25.Source: §4.2 · p. 5, Figure 4 · p. 6

4.3 Jump Sequences

  • Under jump sequences, longer context provides no systematic benefit: ED exceeds 6.5 at β = 0.25 and 13.88 at β = 0.75, with performance sometimes worsening against no history.At β = 0.75, slopes remain far below −10 and the failure is amplified.Source: §4.3 · pp. 5, 7, Figure 5 · p. 6

4.4 Quantitative Analysis Summary

  • With monotonic trends, history reduces equilibrium deviation, but without trends deviation remains high or worsens; β = 0.75 yields worse decision quality than β = 0.25 across trajectories.The tables summarize equilibrium deviation and equilibrium determination across five price trajectories.Source: §4.4 · p. 7, Table 1 · p. 7

5 Related Work

Prior work examines both reasoning-based and pattern-matching accounts of ICL, along with LLM behavior in strategic games. This paper situates its approach among work on recursive reasoning, REE, and game-theoretic evaluation.

5.1 Mechanisms of In-Context Learning in Decision-Making

  • ICL research contrasts implicit reasoning accounts with statistical pattern matching, while sequential decision studies examine adaptation from interaction histories.Accounts include Bayesian inference and in-context optimization, as well as induction-head explanations.Source: §5.1 · pp. 7–8

5.2 Recursive Belief Reasoning and Multi-Agent Games

  • Behavioral game-theory models characterize limited strategic depth, while recent LLM studies assess iterative or recursive strategic reasoning without isolating ICL mechanisms.The cited paradigms include level-k, cognitive hierarchy, beauty contests, self-play, and hypergames.Source: §5.2 · p. 8

5.3 Global Games and Rational Expectations Equilibrium

  • REE is history-independent, providing a rational-play benchmark; using that property to diagnose ICL mechanisms in multi-agent settings remains underexplored.Global games also offer unique equilibria useful for empirical evaluation.Source: §5.3 · p. 8

5.4 LLM Agents in Game-Theoretic Experiments

  • Prior LLM game studies characterize cooperation or competition, whereas jointly manipulating history structure and interdependence offers a complementary way to distinguish reasoning from extrapolation.The comparison builds on REE's history-independence property.Source: §5.4 · p. 8

6 Conclusion

The paper tests whether ICL in multi-agent interdependent settings reflects recursive belief reasoning or statistical extrapolation by manipulating feedback structure in a repeated public goods game against a history-independent REE benchmark. Improvements largely disappear when temporal trends are removed, especially under stronger interdependence.

6 Conclusion

  • The study manipulates feedback-sequence structure in a repeated public goods game and evaluates behavior against a history-independent REE benchmark.The design separates statistical extrapolation from recursive belief reasoning.Source: §6 · pp. 8–9
  • When context lacks a clear temporal trend, ICL's decision-making improvement largely disappears, with the pattern more pronounced at higher interdependence.The observations are consistent with statistical extrapolation rather than recursive belief updating toward equilibrium play.Source: §6 · p. 9
  • The REE-based framework is proposed as a diagnostic tool for studying when LLM agents engage in recursive reasoning; future work could test finer-grained behavior and interventions toward equilibrium-consistent play.These are proposed directions rather than tested outcomes.Source: §6 · p. 9

7 Limitations

The study identifies limits on the scope of its findings, while the appendix derives REE existence and uniqueness conditions and an explicit participant benchmark under uniform private values.

7 Limitations

  • The study tests only two model families, one public-goods interdependence structure, short contexts, behavior-only evidence, and one fixed prompt format.The authors note these boundaries may limit generalization and leave internal mechanisms and framing sensitivity open.Source: §7 · p. 9

A.1 General Existence and Uniqueness Condition

  • When β(n−1)f(θ)<1 for all θ, G is strictly increasing and θ*(p) has a unique solution; the appendix details this REE derivation.Under the paper’s experimental settings, the condition is satisfied and equilibrium always exists and is unique.Source: A Equilibrium Derivation and… · p. 10, §A.1 · p. 11

A.2 Explicit Equilibrium Solution under Uniform Distribution

  • Under uniformly distributed private values, the equilibrium number of participants is linear in public price, with slope and intercept depending on β and the distribution’s support.The participation probability is 1−F(θ*(p)).Source: §A.2 · p. 11

A.3 Normalization Mapping and Unified Evaluation Scale

The appendix maps prices to a unified evaluation scale while preserving the REE’s history independence, then reports supplementary trajectory figures and regression results.

A.3 Normalization Mapping and Unified Evaluation Scale

  • For each β, an affine mapping sends the price interval [pmin(β), pmax(β)] to normalized prices [0, 5], yielding a common equilibrium participant benchmark; experiments use n=50.The mapping bounds are obtained by inverting the equilibrium relation.Source: §A.3 · p. 11
  • The per-period deterministic normalization preserves REE history independence; because equal normalized prices can represent different original prices across β, inference relies on within-group sequence effects and interactions.The appendix presents this normalization as a standardized evaluation framework.Source: §A.3 · p. 11

B.1 Supplementary Results

  • Figures 6 and 7 illustrate decreasing trajectories under monotonic sequences and diverging trajectories under jump sequences; Table 3 gives the full regression analysis.Source: §B.1 · pp. 11–12

B.2 Full Sample Analysis

The full-sample analysis finds that sequential context structure is the main driver of deviations from REE, with interdependence amplifying sensitivity to those patterns. The supplementary workflow and equilibrium sections formalize the experiment and its rational benchmark.

B.2 Full Sample Analysis

  • The full-sample regressions use 7,800 observations, price-path fixed effects, and a Yeo–Johnson transformation for the heavy-tailed deviation outcome.Regressors include price, interdependence intensity, private value, and context window; context levels correspond to 1, 7, and 13 history rounds.Source: §B.2 · p. 12
  • The sequential structure of context is the main force pulling behavior away from REE, while stronger interdependence intensifies its effects rather than replacing it.The main effect of β is small and statistically indistinguishable from zero in baseline specifications, but β interacts strongly with price and trend signals.Source: §B.2 · pp. 13–14

C.1 Experimental Workflow Formalization

  • The workflow formalization represents the experiment as a finite-state machine with states, events, a transition function, and an initial state.The transition function specifies how an event moves the system from its current state to the next.Source: §C.1 · pp. 14–15

C.2 States

  • The five workflow states cover initialization, information broadcast, parallel agent decisions, outcome aggregation and payoff calculation, and termination.Agents decide independently and synchronously using private value, current public cost, and historical data; the environment aggregates actions into participation and payoffs.Source: §C.2 · pp. 15–16

C.3 Events

  • Events mark experiment start, broadcast completion, completion of all decisions, result calculation, and satisfaction of termination conditions.Completion of all agent decisions synchronizes the distributed decision process before centralized results are calculated.Source: §C.3 · p. 16

C.4 State Transition Function

  • The transition function moves initialization to broadcast, broadcast to decisions, and completed decisions to results, then either loops to another round or terminates.The loop occurs when the experiment has not ended; meeting termination conditions moves it to an absorbing final state.Source: §C.4 · p. 16

C.5 Workflow Execution

  • Workflow execution initializes parameters, broadcasts each round’s public cost and prior feedback, collects independent agent decisions, calculates outcomes and payoffs, and repeats until costs run out.Agents form participation expectations from limited information and cannot observe other agents’ live decisions.Source: §C.5 · pp. 16–17

D.1 Equilibrium Solution of the Theoretical Model

  • The REE solution is obtained iteratively by predicting agents’ expectations and behavior, with a unique equilibrium state under the stated assumptions.The calculation adjusts final utility by 0.01 to avoid ambiguous zero-utility participation decisions.Source: §D.1 · p. 17

D.2 Generation of Experimental Price Settings

The experimental price settings are derived from REE participant targets for two interdependence levels. The same prices are then reordered into trend and jump sequences to manipulate historical structure.

D.2 Generation of Experimental Price Settings

  • Under REE, prices decline as participant count rises from 0 to 50: from 49.99 to 12.49 at β = 0.25 and from 49.99 to 37.49 at β = 0.75.Source: §D.2 · p. 17

D.3 Experimental Scenario Design and Price Sequences

  • The experiment reorders the same prices into monotonic increasing and decreasing sequences, each with a single-direction trend.The decreasing order uses indices 0–5, while the increasing order uses indices 5–0.Source: §D.3 · pp. 17–18
  • Jump sequences disrupt a stable trend by alternating high and low prices, either converging from the ends to the middle or diverging from the middle to the ends.The converging order is 5, 0, 4, 1, 3, 2; the diverging order is 2, 3, 1, 4, 0, 5.Source: §D.3 · p. 18

E Experimental Setup

The setup follows model-specific decoding recommendations and tests a lower-temperature condition. A Qwen3-Plus replication shows qualitatively similar trends to GPT-5, alongside quantitative differences in price sensitivity and dispersion.

E Experimental Setup

  • In supplementary analysis, all experiments were rerun with every model’s temperature uniformly set to 0.35, other parameters unchanged; this small randomness was intended to prevent deterministic output loops in complex scenarios.Source: E Experimental Setup · p. 18

F.1 Performance across Different Model

  • Qwen3-Plus qualitatively matches GPT-5: both perform more accurately and consistently under weak network effects, and worse with greater divergence under strong effects.The replication replaces GPT-5 with Qwen3-Plus across the experimental scenarios.Source: §F.1 · p. 18
  • Qwen3-Plus is less sensitive to price trends than GPT-5, with greater deviation of group expected means from equilibrium and systematically higher expected dispersion.The authors suggest this may reflect relative limitations in processing complex contextual information and reasoning.Source: §F.1 · pp. 18–19

F.2 Comprehensive Robustness Results

The robustness analysis lowers GPT-5’s decoding temperature while preserving the other parameter settings. The authors report that the qualitative response patterns remain consistent with the benchmark-temperature results.

F.2 Comprehensive Robustness Results

  • For GPT-5, reducing decoding temperature from benchmark T=0.7 to T=0.35 did not fundamentally alter core decision-making patterns; qualitative responses across network-effect strengths and historical-window lengths remained highly consistent with benchmark results.Other parameter configurations were maintained at the optimal parameter set described in Appendix E.Source: §F.2 · p. 19
  • The Qwen3-Plus robustness table covers equilibrium deviation and equilibrium determination coefficient across trajectories.The supplied table caption identifies the measures and scope but does not provide cell values.Source: Table 5 · p. 19
  • The temperature-0.35 figures show monotonic decreasing and increasing, converging-jump, and diverging-jump price trajectories for GPT-5.The figure captions identify the trajectories but do not state comparative outcomes.Source: Figure 12 · p. 22, Figure 13 · p. 22, Figure 14 · p. 23

Related papers

Loading 2609.18591v1…