PaperlayerView exact revision on arXiv

Source-linked AI summary

When LLM Agents Fail to Read the Room: ReAdapt for Relational Social Reasoning

Jianzhe Lin, Xiaolin Li, Yunda Liu, Fei Wang, Jubin Chheda

arXiv:2609.25284v1cs.AI

TL;DR

Social decisions often depend on relationship evidence, but standard agent loops do not explicitly represent how observations revise a social hypothesis. ReAdapt adds structured state adaptation to the agent loop, and improves accuracy on both relationship-reasoning tasks in a controlled evaluation.

  • Problem

    Standard agent loops do not require an explicit transition from a current social hypothesis to a revised one when new relational evidence arrives.

    Source: §1 · p. 3

  • Method

    ReAdapt extends ReAct with a structured social state that is updated after observations and paired with a typed policy operation.

    Source: §1 · p. 3

  • Results

    In a stratified 300-query Gemini-3-Flash evaluation, warm-introduction accuracy rises from 37% to 51% and reaction-selection accuracy from 69% to 77%.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The results suggest explicit relational-state adaptation can improve relationship-grounded decisions on reaction selection and warm introduction.

    Source: §6 · p. 10

    The current experiments primarily exercise belief and relationship dimensions and do not independently validate the necessity of all five state dimensions.

    Source: §3.2 · p. 7

Abstract

from arXiv · show

A social agent's most basic decisions (should I react to this post? who should I reach out to?) are not purely content problems. The right action often hinges on the latent relationship between people -- tie strength, reciprocity, mutual connections -- rather than on which content is most salient. Standard LLM agent loops do not explicitly represent how new relational evidence should revise the agent's current social hypothesis, leaving them prone to surface-obvious choices when relational and content cues diverge. We formalize this failure mode with a relationship-reasoning benchmark: 500 synthetic social worlds with friendships, follows, reaction histories, and feeds, yielding 1,000 queries over two tasks, reaction selection and warm introduction (finding the best bridge to a target person). By construction, the surface-obvious candidate differs from the relationship-grounded oracle in about 53% of queries, forming an overturn subset where the agent must use relational evidence to revise an initially plausible choice. We propose ReAdapt (Relationship-Adaptive Agent with Policy-driven sTate), which augments the ReAct loop with an explicit structured social state z = (G, B, R, N, D) capturing goal, belief, relationship, norm, and disclosure. After each tool observation, ReAdapt runs a typed Adapt step that updates this state and emits a policy operation (continue, switch, abandon, or clarify) before choosing the next action. With Gemini-3-Flash on a stratified subset of n = 150 queries per task, ReAdapt improves warm-introduction accuracy from 37% to 51% (+14 points) and reaction-selection accuracy from 69% to 77% (+8 points). Oracle regret drops from 0.260 to 0.152 and from 0.095 to 0.053, respectively. Holding the model, tools, and environments fixed, these results suggest that explicit relational-state adaptation helps LLM agents turn retrieved social evidence into revised decisions.

1 Introduction

Social decisions depend on relationship history and norms, not just salient content, yet standard agent loops leave relational evidence’s effect on a current hypothesis implicit. ReAdapt makes that revision explicit through structured state and policy operations, and improves accuracy on both benchmark tasks.

1 Introduction

  • Retrieving evidence and revising a decision are distinct: standard ReAct leaves an observation’s influence on a social hypothesis implicit in language-model context.The benchmark targets cases where an agent acquires relevant information but does not reliably let it alter its policy.Source: §1 · p. 3
  • The benchmark yields 1,000 queries across reaction selection and warm introduction, and approximately 53% of the full benchmark are overturn cases, where the surface-obvious candidate differs from the relationship-grounded oracle.Source: §1 · p. 3
  • ReAdapt maintains a structured social state for goal, belief, relationship, norm, and disclosure, then adapts it after observations and emits a typed policy operation.Operations are continue, switch, abandon, or clarify.Source: §1 · p. 3
  • ReAct and ReAdapt receive the same external evidence; their distinction is implicit evidence integration versus an explicit, policy-relevant state transition.ReAdapt addresses whether contradictory evidence changes the current social hypothesis before acting.Source: §1 · p. 3
  • ReAdapt improves warm-introduction accuracy from 37% to 51% and reaction-selection accuracy from 69% to 77%, with the largest observed gain on warm-introduction overturn cases, from 33% to 49%.Source: §1 · p. 4
  • Mean benchmark-oracle regret falls from 0.260 to 0.152 on warm introduction and from 0.095 to 0.053 on reaction selection.Regret measures the severity of an error relative to the benchmark oracle, not universal human social utility.Source: §1 · p. 4

2 Relationship-Reasoning Benchmark

The benchmark uses synthetic social worlds to test reaction selection and warm introductions, where relational evidence may conflict with an initially salient choice. Its overturn diagnostic isolates queries where success requires gathering relational evidence and revising that choice.

2.1 Synthetic Social Worlds

  • The 500 procedurally generated worlds vary graph structure, density, reciprocity, interaction history, and alignment between surface-level and relational evidence.Each world has 8–15 users and a designated ego; the generator has full access to the environment, while agents must retrieve relevant evidence through tools.Source: §2.1 · p. 4

2.2 Tasks

  • Reaction selection scores candidate posts using relational features and content affinity, while warm introduction selects an intermediary based on relationships and shared context along the ego–bridge–target path.Warm introduction requires reasoning over indirect relational structure rather than only a direct ego–author tie.Source: §2.2 · pp. 4–5

2.3 The Overturn Diagnostic

  • About 53% of queries are overturn cases, where the surface heuristic is insufficient and success requires gathering relational evidence and revising an initially plausible choice.Reporting overturn and straight subsets helps distinguish general task competence from the ability to revise a misleading initial hypothesis.Source: §2.3 · p. 5

2.4 Tool Interface

  • Agents investigate each world through the same seven tools, which expose profiles, friendships, follows, reaction histories, and post content, with at most 24 calls per episode.Both agents have access to the same environment; the challenge is deciding what evidence to retrieve and how to use it.Source: §2.4 · p. 5

2.5 Evaluation Metrics

  • Evaluation reports pick accuracy, overturn accuracy on cases where the surface candidate conflicts with the oracle, and oracle regret, which captures how close an incorrect prediction is to the optimum.Lower regret is better.Source: §2.5 · p. 6

3 Method: ReAdapt

ReAdapt adds an explicit state-update step to the tool-using agent loop, requiring retrieved evidence to modify a persistent representation before subsequent action. The method distinguishes this policy-relevant transition from merely receiving or reasoning over the same observations.

3 Method: ReAdapt

  • ReAdapt augments the observation–action loop with an Adapt stage that explicitly updates the agent’s current model of the social situation before its next decision.The design makes newly retrieved observations updates to state rather than merely additional text in the interaction history.Source: §3 · p. 6

3.1 Baseline: ReAct

  • Standard ReAct leaves the integration of observations and any resulting revision of the agent’s relational hypothesis to free-form reasoning.Relevant evidence may be distributed across multiple observations, such as friendship, reciprocity, and interaction history.Source: §3.1 · p. 6

3.2 Structured Social State

  • The structured social state represents goal, belief, relationship, norm, and disclosure, but the benchmark primarily exercises belief and relationship and does not independently validate the other dimensions.Relationship includes tie strength, reciprocity, and interaction evidence; disclosure concerns constraints on what information may appropriately be revealed.Source: §3.2 · pp. 6–7

3.3 Adapt Before Acting

  • The Adapt output records a concise state assessment and its change from the previous state, then emits a typed policy operation that conditions the next action.The central distinction is that newly retrieved evidence must explicitly modify a persistent representation before subsequent action.Source: §3.3 · p. 7

3.4 From Evidence Retrieval to Hypothesis Revision

  • The switch operation is intended to make evidence-driven revision explicit when new relational evidence favors a candidate over an initially surface-salient choice.The benchmark’s overturn subset provides a diagnostic for this behavior.Source: §3.4 · p. 7

3.5 Implementation

  • The comparison holds model, query, candidates, world, tools, and maximum tool-call budget constant, but ReAdapt generates additional structured reasoning tokens.The experimental intervention is the reasoning protocol: ReAdapt maintains state, updates after each observation, and selects a policy operation.Source: §3.5 · p. 8

4 Experiments

In a controlled comparison on a stratified 300-query subset, ReAdapt improves accuracy on both tasks, with its largest gain on warm-introduction overturn cases. Warm introduction is harder for both agents, and the task comparison is consistent with—but does not causally isolate—relational depth as an explanation.

4.1 Experimental Setup

  • The evaluation uses Gemini-3-Flash on 300 stratified queries, including 150 per task and both overturn and straight cases; both conditions share the model, queries, tools, and a 24-call maximum.The comparison is described as a controlled proof of concept, not an exhaustive evaluation of all 1,000 queries.Source: §4.1 · p. 8

4.2 Main Results

  • Warm-introduction accuracy rises from 37% to 51%, while reaction-selection accuracy rises from 69% to 77% with ReAdapt.These are absolute gains of 14 and 8 percentage points, respectively.Source: §4.2 · p. 8
  • Warm-introduction overturn accuracy rises from 33% to 49%; reaction-selection overturn accuracy rises from 70% to 76%.The 16-point warm-introduction gain is the largest observed improvement; the reaction-selection gain is 6 points.Source: §4.2 · p. 8
  • Straight-case accuracy also improves: 42% to 54% for warm introduction and 68% to 78% for reaction selection.The reported benefit is therefore not restricted to cases requiring reversal of a surface-level choice.Source: §4.2 · p. 8
  • Oracle regret falls from 0.260 to 0.152 on warm introduction and from 0.095 to 0.053 on reaction selection.The reductions are 41.5% and 44.2%, respectively; selected candidates tend to lie closer to the benchmark optimum even when not top-ranked.Source: §4.2 · p. 9

4.3 Task Difficulty

  • Warm introduction is harder for both agents: ReAct reaches 37% accuracy there versus 69% on reaction selection.Relational depth is a plausible explanation because warm introduction integrates ego–bridge and bridge–target relationships, but the experiment does not causally isolate graph depth.Source: §4.3 · p. 9

5 Analysis

The analysis illustrates how ReAdapt externalizes revisions when retrieved relational evidence favors an alternative to an initially plausible choice. It also reports modestly higher tool use, so the comparison is not one of equal computational cost.

5.1 From Retrieved Evidence to Revised Decisions

  • In inspected failures, ReAct can mention evidence favoring an alternative yet retain its initial choice; ReAdapt requires an explicit state change and policy update.The illustrative overturn trajectory runs from a plausible surface cue through stronger relational evidence and a state update to switching and then investigating or confirming the alternative.Source: §5.1 · p. 9
  • The trajectory analysis is illustrative, not a population-level causal decomposition, and the experiment does not isolate individual state dimensions or policy operations.Source: §5.1 · p. 9

5.2 Tool Usage

  • ReAct averages 8.2 tool calls per episode versus 9.4 for ReAdapt, with both below the shared maximum of 24.The systems are not perfectly compute matched; the comparison is framed as reasoning protocols under matched model and environment access, not equal computational cost.Source: §5.2 · p. 9

6 Conclusion

The paper introduces ReAdapt, which adds explicit social-state updates and typed policy operations to the ReAct loop, alongside a controlled relationship-reasoning benchmark. In a stratified 300-query evaluation, ReAdapt improves accuracy on both tasks, with its largest gain on warm-introduction overturn cases.

6 Conclusion

  • ReAdapt augments ReAct with explicit structured social-state updates and typed policy operations, alongside a benchmark of 500 synthetic social worlds and 1,000 queries across two tasks.Source: §6 · p. 9
  • In a stratified 300-query evaluation, accuracy rises from 37% to 51% on warm introduction and from 69% to 77% on reaction selection.The largest observed gain is 16 percentage points on warm-introduction overturn cases; benchmark-oracle regret also decreases substantially on both tasks.Source: §6 · p. 10
  • The results support making social observations explicit state changes rather than allowing them to accumulate only in context.ReAdapt is presented as one lightweight mechanism for that transition.Source: §6 · p. 10

A Benchmark Sanity Checks

Scripted responders check that the evaluation harness implements the intended relationship between the oracle answer, surface candidate, and overturn flag. The authors characterize these as software-level sanity checks, not validation that the oracle captures real-world social preferences.

A Benchmark Sanity Checks

  • The oracle responder achieves 100% pick accuracy, while the surface responder achieves 100% on straight cases and 0% on overturn cases.Source: A Benchmark Sanity Checks · p. 11
  • These scripted checks validate evaluation-code behavior, not whether the oracle captures all dimensions of real-world social preference.Source: A Benchmark Sanity Checks · p. 11

B Related Work

Prior agent research emphasizes reasoning, memory, and social simulation, while ReAdapt focuses on whether a single user-facing agent can revise a relationship-grounded decision as evidence arrives.

B Related Work

  • Unlike work on emergent population behavior, this setting tests whether one agent can make relationship-grounded choices when surface and relational cues conflict.Generative Agents demonstrated coherent long-horizon social behavior using memory, reflection, and planning.Source: B Related Work · p. 11
  • ReAdapt complements ReAct, Reflexion, and cognitive-agent approaches by specializing agent state for relational decisions and requiring explicit state changes before later actions.It does not introduce a general-purpose search or memory algorithm.Source: B Related Work · p. 11
  • Rather than test isolated social-question answering, ReAdapt studies whether accumulated tool evidence revises an action; it draws on established computational models of ties using interaction and network features.Prior work links online tie strength to behavioral and network features.Source: B Related Work · p. 11
  • ReAdapt makes relational evidence an active decision problem: the agent must discover relevant evidence through tools and let it influence a subsequent choice.The contribution is not a new estimator of human tie strength.Source: B Related Work · p. 12

C Limitations and Scope

The study evaluates a specified relationship-grounded objective in synthetic worlds and a controlled subset of queries, with one model and no component-level ablation.

C Limitations and Scope

  • Synthetic worlds provide control over latent structure and avoid privacy concerns, but simplify the ambiguity and idiosyncrasy of real interpersonal decisions.The benchmark oracle operationalizes a study-specific objective rather than a universal definition of socially optimal behavior.Source: C Limitations and Scope · p. 12
  • The frontier-model evaluation uses a stratified subset of 300 of the benchmark’s 1,000 queries, so results are a controlled proof-of-concept comparison, not a complete full-benchmark performance characterization.Source: C Limitations and Scope · p. 12
  • Only Gemini-3-Flash was evaluated, and each query ran once at temperature 0, leaving cross-model effects and run-to-run variance unmeasured.Hosted inference may retain residual nondeterminism.Source: C Limitations and Scope · p. 12
  • The ReAct comparison tests the complete ReAdapt protocol, not the separate contributions of relationship state, state deltas, policy operations, or added structured reasoning.ReAdapt also adds an Adapt block after observations and uses slightly more tool calls on average, so computation is not matched at the token level.Source: C Limitations and Scope · p. 12
  • The current tasks primarily test belief and relationship state; the independent value of goal, norm, and disclosure dimensions remains unevaluated.Source: C Limitations and Scope · p. 12

D Discussion

The discussion distinguishes access to relational evidence from using it to revise a decision, and frames explicit state adaptation as an interface for that revision.

D Discussion

  • Both ReAct and ReAdapt can retrieve friendship, follow, reciprocity, and interaction signals, but their availability does not guarantee revision of an already plausible choice.Source: D Discussion · p. 12
  • On overturn cases, success requires hypothesis revision; ReAdapt updates state from observations, can emit switch, and conditions the next action on revised state.Source: D Discussion · p. 12
  • The discussion frames social reasoning as more than retrieval: tool observations are updates to evolving social state rather than additional unstructured context.Source: D Discussion · p. 12
  • The broader personal-agent vision includes persistent models of goals, relationships, norms, and disclosure boundaries, but deployment would also require uncertainty calibration, privacy safeguards, temporal adaptation, user correction, and human-outcome evaluation.The present benchmark covers only a controlled subset of that broader problem.Source: D Discussion · p. 13
Loading 2609.25284v1…