PaperlayerView exact revision on arXiv

Source-linked AI summary

Delay-of-Gratification as a Multi-Agent Survival Micro-benchmark for Long-Horizon LLMs: Social Exposure, Personas, and Tool Use Budgets

Olga Manakina, Igor Bogdanov, Chung-Horng Lung

arXiv:2609.29509v1cs.AIcs.CL

TL;DR

Existing research lacks auditable, multi-factorial evidence on how LLM behavior unfolds over long, multi-turn interactions. This paper builds a minute-by-minute marshmallow-inspired benchmark that varies social context, personas, and tool policy, and finds that isolation lowers eating hazard while mandatory self-questioning raises it.

  • Problem

    Existing evaluations provide limited auditable, constrained evidence on how LLM behavior unfolds over time and in the presence of other agents.

    Source: §1 · p. 2

  • Method

    The paper evaluates LLM agents in a marshmallow-inspired survival benchmark that varies social context, personas, and tool policy and analyzes outcomes with KM curves and discrete-time hazard models.

    Source: §4.1 · p. 6, §7 · p. 9

  • Results

    Isolation lowers per-minute hazard vs. broadcast (OR ≈0.78), while mandatory self-questioning raises hazard (OR ≈1.42).

    Source: §6 · p. 9

  • Takeaways & Limitations

    The benchmark offers a compact testbed for examining how social exposure and metacognitive scaffolding relate to temporal decisions in LLM agents.

    Source: §6 · p. 9, §7 · p. 9

    Evidence is scoped to Gemini 2.5 Flash-Lite with fixed decoding and one binary-action micro-environment; cross-model and decoder generalization remains unknown.

    Source: §6 · p. 9

Abstract

from arXiv · show

Large language models (LLMs) are increasingly deployed as multi-turn agents that must sustain goals, use tools, and adapt to other agents over extended interactions. However, existing research lacks auditable, multi-turn, multi-factorial experiments that quantify LLM behavior under explicit constraints, with time-resolved statistics that reveal how behavior unfolds over long horizons. To address this gap, we develop a multi-agent micro-benchmark inspired by the Stanford marshmallow experiment: ReAct agents operate minute-by-minute with a "raise a question" tool under a per-step budget, while we factorially manipulate social context (broadcast vs. isolated), personas (age, hedonic drive), and metacognitive policy (mandatory vs. optional tool use). We analyze outcomes with Kaplan-Meier (KM) survival curves and discrete-time hazard models over a long risk horizon across 19,200 agent trajectories in 64 cells. Behavior shows a sharp early "eat" impulse, and only 75.9% of agents persist to the end. In a discrete-time hazard model, isolation reduces per-minute risk relative to broadcast, whereas a must-use self-questioning policy increases risk. On average, agents ask $\approx 7.12$ questions and hit the per-step budget in $\approx 6\%$ of minutes. Questioning declines faster under broadcast than isolation. Ablation experiments demonstrated that removing hedonic drive and/or persona age increases survival and completion, narrows the broadcast/isolated gap, but leaves the must vs. may ordering intact. The combined ablation (no hedonic + no persona age) yields the highest completion (approaching $1.0$). These results establish delay-of-gratification as a compact, multi-turn interaction benchmark that captures social contagion and tool-use dynamics in LLM agents, providing a reproducible testbed and statistics for analyzing long-horizon, multi-agent behavior.

1 Introduction

The introduction motivates auditable, constrained studies of multi-turn LLM behavior and tests how social visibility, personas, and metacognitive scaffolding shape waiting. Results show that social visibility and mandatory self-questioning increase risk, while removing persona components improves survival and completion.

  • Existing multi-turn evaluations motivate auditable, constrained experiments because single-turn performance does not guarantee long-horizon reliability.Prior work also reports interaction-specific effects of tools and feedback, and performance drops from single- to multi-turn settings.Source: §1 · p. 2
  • The study tests whether social visibility, hedonic drive and age, mandatory self-questioning, time dependence, and prompt scaffolding affect eating hazard.The preregistered expectation that prescriptive prompts and enforced metacognition would improve adherence contrasts with the reported MUST result.Source: §1 · p. 2
  • Across 19 minutes and 19,200 trajectories, isolation lowers per-minute hazard vs. broadcast (OR ≈0.78), while MUST increases hazard (OR ≈1.42).Removing hedonic drive and/or persona age improves survival and completion, narrows the social-context gap, and the combined ablation approaches perfect completion.Source: §1 · p. 2

2 Related work

The paper situates its benchmark among cognitive-task adaptations, delay-of-gratification research, and work on LLM interaction design. It contributes a controlled survival-analysis framework that tests social exposure, persona prompts, and tool-use policy over extended interactions.

2 Related work

  • Prior LLM cognitive-task studies are largely single-prompt or short-horizon, and an earlier LLM marshmallow study lacked temporal and social analyses.This work instead studies delayed gratification in a controlled, minute-by-minute, multi-agent setting.Source: §2 · p. 2
  • The paper adapts delay-of-gratification and intertemporal-choice ideas to discrete-minute survival analysis, drawing on interaction, tool-use, multi-agent, and persona-evaluation research.The related work also situates the analysis in KM and discrete-time logistic hazard methods.Source: §2 · pp. 2–3

2.1 Contributions

  • The contribution reframes marshmallow-like choice as a long-horizon survival task and tests social context, personas, and tool policy with survival and hazard models.Peer visibility and child-like or craving personas elevate early-eating risk, while mandatory tool use also increases hazard.Source: §2.1 · p. 3

3 Experiment Setting

The experiment models waiting as a finite-horizon, minute-by-minute interaction in which agents choose to wait or eat, with observations shaped by social context. It varies social exposure, personas, and tool-use policy, and records trajectories for survival-based analysis and reproducible evaluation.

  • Agents act in a turn-based ReAct loop, using observations and optional internal deliberation before choosing to wait or eat; eating ends their participation.Broadcast observations include recent peer actions, while isolated observations contain the agent’s own history and make the environment fully observable.Source: §3 · p. 3, Figure 1 · p. 4
  • The factorial design varies isolated versus broadcast context, hedonic drive and age personas, and mandatory versus optional tool use, then evaluates outcomes with KM curves and hazard models.The persona options include child, adult, senior, or none for age, and crave, like, neutral, or none for hedonic drive.Source: §3 · p. 4
  • The environment models 20 minutes of delay gratification as T discrete steps (default T = 20, Step 0 initialization), rewards eating at any minute t with +1.0 followed by elimination, and rewards persistence through the final minute with +2.0.Source: §3 · p. 4
  • The raise_a_question tool supports internal self-querying under a per-step budget; agents either MUST use it or MAY use it, and the budget cap may be visible or hidden.The tool is internal and does not alter the environment state.Source: §3 · pp. 3–4
  • The run harness records full ReAct traces, per-agent outcomes, timing, rewards, and tool-use patterns for downstream analysis and reporting.The implementation separates abstractions from scenario plugins and produces analysis-ready data and reports.Source: §3 · pp. 4–5

4 Methods and Procedure

The study crosses social context, persona, and tool-use policy in a minute-by-minute agent experiment, then uses survival and hazard analyses alongside descriptive measures of social signals and tool use.

4 Methods and Procedure

  • The factorial design varies social context, hedonic drive, age persona, and MUST versus MAY tool-use policy, with agents following a per-minute ReAct loop and a capped question tool.Agents use Gemini 2.5 Flash-Lite across cells and trials.Source: §4 · p. 5
  • The procedure initializes agents, runs a minute-by-minute loop for active agents, and resolves the final minute by having remaining agents commit to waiting for +2.0.The pipeline logs decisions, rewards, tool use, validation, social exposure, and run metadata.Source: §4 · p. 5

4.1 Metrics and Statistical Analysis

  • The event is an agent’s first “I eat the marshmallow” response; agents who do not eat by horizon T are right-censored and coded as “waited_full.”Invalid format-violation steps are tracked and excluded under pre-specified rules.Source: §4.1 · p. 5
  • Restricted mean survival time summarizes the area under the survival curve through τ minutes, with condition-wise estimates and differences receiving 95% bootstrap confidence intervals clustered by trial.The minute-level estimate uses the KM survival probability at the start of each minute.Source: §4.1 · p. 5
  • Kaplan-Meier curves estimate the probability of continuing to wait at each minute, with uncertainty represented by 95% confidence intervals using Greenwood’s formula.The curves are plotted for experimental factors such as communication mode and hedonic drive.Source: §4.1 · p. 6
  • A logistic-regression discrete-time hazard model estimates the conditional probability of eating at time t from time dummies and experimental-condition covariates.Time dummies capture baseline changes over time, while covariate coefficients quantify effects on log-odds of eating; descriptive plots also track peer signals and questions asked by step.Source: §4.1 · p. 6

5 Results

Across 19,200 trajectories, agents showed a strong early impulse to eat and 75.9% persisted to the end; social context, personas, and tool policy differentiated survival. Ablations improved survival and completion, while question use varied by social context.

5 Results

  • 19,200 trajectories across 64 cells showed an initial eat rate ≈0.125 and winners rate ≈0.759, with a strong first-minute impulse followed by a long tail of waiting.The median time-to-eat and RMST were both ≈14.8 minutes.Source: §5 · p. 6

5.1 Main Effects

  • Isolation lowered eating hazard versus broadcast (OR ≈0.78), while MUST raised it versus MAY (OR ≈1.42).Both effects were statistically significant (p < 0.001).Source: §5.1 · p. 6
  • Compared with crave, like (OR ≈0.28), none (OR ≈0.19), and neutral (OR ≈0.03) significantly lowered eating hazard (all p < 0.001); child (OR ≈66.3, p < 0.001) and senior (OR ≈7.55) had elevated hazard versus adult.Source: §5.1 · p. 6

5.2 Interaction Dynamics

  • Question-asking, a proxy for deliberation, declined faster under broadcast than isolation.Agents asked ≈7.12 questions on average and hit the per-step budget in ≈6% of minutes.Source: §5.2 · p. 7, §5.3 · p. 8, Figure 4 · p. 8

5.3 Ablations

  • Removing hedonic drive increased survival in both social contexts, removing persona age shifted survival further upward, and removing both narrowed the broadcast-isolated gap.The combined ablation produced the highest survival, with curves nearly flat after the early minutes.Source: §5.3 · p. 8, Figure 5 · p. 8
  • Across ablations, MAY improved completion by ≈3.5 percentage points over MUST (winners 0.7761 vs. 0.7411).MAY also had lower total eat rate (0.2232 vs. 0.2580) and higher RMST (15.11 vs. 14.49).Source: §5.3 · p. 8
  • Completion rates rose monotonically from full factors through the combined ablation, approaching 1.0 under the combined ablation in both social contexts.Figure 5b displays this completion-rate progression.Source: §5.3 · p. 8, Figure 5 · p. 8

6 Discussion

The discussion frames early temptation followed by low-hazard persistence as a stress test for long-horizon reliability. It identifies social exposure and mandatory tools as factors associated with increased risk.

6 Discussion

  • Early temptation followed by low-hazard persistence makes the framework a stress test for long-horizon reliability.Source: §6 · p. 8
  • OR ≈0.78 for isolated agents accompanies lower give-in hazard, while broadcast visibility accelerates the decline in self-questioning.The authors suggest that peer observation reshapes both decisions and reasoning processes.Source: §6 · p. 9
  • Mandatory tool use increases hazard (OR ≈1.42), and front-loaded deliberation may focus attention on temptation at critical decision points.Source: §6 · p. 9
  • Evidence is bounded by one model and decoding setup, a binary-action fixed-reward task, and only isolated-versus-broadcast social contexts.Question-budget visibility was held hidden, and omitted time-varying covariates limit causal claims about social cascades.Source: §6 · p. 9
  • For systems requiring sustained adherence, the authors recommend managing social exposure, preferring optional tools, and monitoring step-level metrics for early impulses.Source: §6 · p. 9

7 Conclusion

The paper presents a controlled, long-horizon benchmark for multi-turn LLM agents, with auditable time-resolved traces. Results indicate that social visibility and mandatory metacognition increase risk, while persona factors and time dependence shape waiting behavior.

7 Conclusion

  • Broadcast visibility increases early-eat hazard, mandatory metacognition raises risk, persona factors modulate waiting, and decisions show an early spike followed by a long tail.Source: §7 · p. 9
  • More prescriptive prompt scaffolding does not improve adherence and can degrade reliability in this setting.Source: §7 · p. 9
  • The marshmallow-inspired benchmark manipulates social contexts, personas, and tool policies, then analyzes auditable traces with Kaplan–Meier survival and discrete-time hazard models.The environment is formalized as an MDP when isolated and a POMDP under broadcast.Source: §7 · p. 9
  • Future work will test broader model families, randomized social schedules for causal leverage, and tasks involving tool budgets and coordination.Source: §7 · p. 9
Loading 2609.29509v1…