PaperlayerView exact revision on arXiv

Source-linked AI summary

AURA: Intent-Directed Probing for Implicit-Need Surfacing in Situated LLM Agents

Yang Li, Jiaxiang Liu, Jiang Cai, Mingkun Xu

arXiv:2606.05557v1cs.CL

TL;DR

Situated agents may answer a user’s literal query while missing an implicit need encoded in hidden private state. AURA inserts an IntentFrame-based inference stage before tool use to route targeted probes, improving implicit-need coverage while operating within a bounded scope where private state is tool-mediated and extractable.

  • Problem

    Situated agents can miss implicit information needs behind literal environmental queries, such as whether another person is available, because relevant context may reside in private state.

  • Method

    AURA uses an LLM-generated IntentFrame and scalar gap score between scene perception and reasoning to control per-query probe budgets and tool selection.

  • Results

    Δ=+0.07 implicit-need coverage over ReAct-style probing on a 100-query four-scene benchmark, with p<10^-6 and replication across scenes, a second backbone, and prompt ablation.

  • Takeaways & Limitations

    AURA supports intent-directed probing as a control point for surfacing hidden private-state context, rather than as a universal accuracy improvement across factual-grounding regimes.

  • Takeaways & Limitations

    The controller is scoped to situated queries with tool-mediated hidden state; cross-domain checks found no measurable lift when private state was already in context or structurally inaccessible.

Abstract

from arXiv · show

A situated query like "where is Lin Wei?" often encodes more than its literal content: the user may also want to know whether Lin Wei is free, in a good mood, or worth interrupting now. Standard tool-use agents answer the literal question and stop. AURA inserts an inference step between scene perception and tool use that produces an IntentFrame: a structured estimate of the implicit need with a scalar gap score that controls per-query probe budget and tool selection. On a 100-query four-scene implicit-intent benchmark, AURA improves implicit-need coverage over ReAct-style probing (Delta = +0.07, p < 10^-6); three of four scenes are individually significant, the gain reproduces on a second backbone, and a prompt ablation attributes the lift to gap calibration rather than answer memorisation. On factual lookup the controller trades raw accuracy for 82% fewer probes and zero forbidden-tool violations on a privacy-sensitive slice; scope conditions are detailed in Limitations. Code, simulator, and benchmark are released at https://github.com/innovation64/AURA.

1 Introduction

AURA addresses the gap between literal situated queries and users’ implicit information needs by making intent inference an explicit control stage for targeted private-state probing. Its benchmark results support improved implicit-need coverage, while the paper defines a narrower scope for settings with tool-mediated hidden state.

  • Motivation: Situated agents can answer literal environmental questions while missing implicit needs such as another person’s availability, leaving decisions and responses insufficiently grounded.The paper frames implicit-need surfacing as a problem of translating environment state into the context needed for the user’s next action.
  • Approach: AURA adds an IntentFrame before tool use to estimate the literal–implicit gap and route private-state probes, tool selection, and probe budget.The gap score controls Explore, while later reasoning and response stages consume the enriched context and probe trace.
  • Positioning: AURA’s contribution is selective access to hidden private state rather than passive inclusion of all environment observations or tool calls driven only by the surface query.The paper contrasts its per-query access control with reactive tool use, literal-query planning, and passive context injection.
  • Scope: The approach targets structured environments where private state is partitioned behind tool-mediated access, not embodied navigation, open-world exploration, or real-time sensor fusion.Its useful range depends on residual uncertainty after passive perception and structurally extractable tool returns.
  • Evidence: Δ=+0.07 implicit-need coverage over ReAct-style NoIntent on 100 queries across four scenes, with p<10^-6 and three of four scenes individually significant.The improvement reproduces on a 25-query pilot, a second backbone, and under disjoint-example ablation.
  • Approach: AURA’s selective probing is bounded rather than world-sized: the Explore loop operates before answer generation with a finite probe budget and a compact probe trace.The supplied passages describe probing as occurring before reasoning and being capped by the gap-derived budget and global maximum.

4 AURATown: A Multi-Agent Social Simulation

AURATown evaluates AURA across environment access, intent-directed probing, and adaptive probe allocation. Results show strong implicit-need gains, while factual lookup is primarily an access-cost and privacy trade-off rather than an accuracy win.

  • Environment and evaluation: AURATown is a 60×60 grid-based social simulation with 5 named agents, 20 named locations, and a public/private state split.Location and action are public, while availability, emotional state, and other private fields require probes.
  • Environment and evaluation: AURA’s factual-grounding evaluation uses 50 environment queries across three seeds, with factual accuracy judged against current environment state.The setup includes spatial, social, temporal, memory, and planning queries, alongside vanilla and tool-using reference conditions.
  • Factual grounding: 1.40 probes per query gives GapRouted an access-cost Pareto position, while Fixed-Probe reaches higher factual accuracy at 8.00 probes per query.GapRouted’s factual accuracy is 0.696 versus Fixed-Probe’s 0.766, so it is not the raw-accuracy winner.
  • Factual grounding: 0% forbidden-tool violations makes gap routing a privacy benefit on the 30-query sensitive slice, compared with 78.9% for Plan-and-Solve and 100% for Fixed-Probe.Low-gap queries receive B=0 and skip the probe registry, structurally preventing forbidden-tool calls.
  • Implicit-intent surfacing: 0.804 implicit-need coverage exceeds 0.733 for ReAct-style NoIntent and 0.223 for Literal on the 100-query four-scene benchmark.The paired gain is ∆=+0.071 with p=1.0 × 10−6; three of four scenes are individually significant.
  • Implicit-intent surfacing: ∆=+0.29 is the largest availability gain, while appropriateness improves by +0.11 and mood by +0.07; latent_goal declines by −0.09.Second_order ties because its surface wording already cues belief-state probing.
  • Mechanism and robustness: 0.803 to 0.368 after replacing LLM gap inference with a heuristic localizes the lift to LLM-mediated gap inference.Disjoint prompt examples reduce Intent by only 0.037, whereas removing examples collapses gap calibration and makes the contrast non-significant.

6 Conclusion

AURA places an IntentFrame between scene perception and tool use to route private-state probes from an inferred literal–implicit gap. It improves implicit-need coverage, but its scope remains bounded to tool-mediated hidden-state settings.

  • Conclusion: AURA inserts an IntentFrame between scene perception and tool use, using its gap score to route private-state probes before answering.The controller improves implicit-need coverage over ReAct-style probing by ∆=+0.07 with p<10−6 on a 100-query benchmark.
  • Conclusion: The lift reproduces across a 25-query pilot, a second backbone, and prompt ablation, while heuristic replacement reduces implicit score from 0.803 to 0.368.This pattern localizes the effect to LLM-mediated gap calibration rather than answer-template memorisation.
  • Conclusion: AURA’s scope is bounded to situated regimes with tool-mediated hidden state, leaving generalisation to multi-turn, multi-user, and open-ended planning settings open.The current IntentFrame operates on a single user query.

Limitations

The controller’s evidence is scoped to situated queries with tool-mediated hidden state and reflects a hybrid architecture. Calibration, evaluation uncertainty, and backbone compatibility constrain how broadly the results should be interpreted.

  • Regime scope: The controller targets situated queries with tool-mediated hidden state, not cases where private state is already in context or structurally inaccessible.Cross-domain checks found no measurable lift on FANToM, LoCoMo, and GAIA in those conditions.
  • Calibration: Removing few-shot calibration examples reduces the gain to non-significance, although benchmark-disjoint replacements preserve it.This makes gap calibration load-bearing for the reported improvement.
  • Evaluation: Human evaluation uses N=8 raters with Krippendorff’s α=0.43; directional agreement is 74%, while magnitude estimates remain substantially uncertain.The benchmark’s five-subcategory partition has κ=0.61 from two independent annotators.
  • Backbone compatibility: Three of four tested backbones reproduce the gain, but gemini-2.5-flash fails the IntentFrame JSON parser and falls back silently to a heuristic.This limits the interpretation of cross-backbone robustness.
  • Architecture boundary: The adaptive-budget result concerns one LLM-controlled point in a hybrid pipeline, while Perceive, Scene, and Memory remain code-determined.Intent inference, probe selection, and response generation use LLM calls.

A3 Private-State Evolution Rules

AURATown’s private-state rules update hidden agent attributes deterministically from actions and context. Beliefs about others refresh only under co-location, creating stale-belief queries for second-order evaluation.

  • State evolution: Private fields update each simulation tick under a deterministic, ordered rule table whose first matching rule wins.The fields include availability, emotional_state, unspoken_goal, and beliefs_about_others.
  • Belief updates: Beliefs about others refresh only when the believer and target are co-located in the same simulation tick.Non-co-located beliefs retain their most recently observed value and become stale when peers move.
  • Benchmark implication: Second-order queries require the believer’s recorded belief rather than the target’s current ground truth.This makes belief staleness part of the benchmark substrate.
  • Reproducibility: The pure-function evolution is covered by 14 unit-tested cases, including all rules and co-location/staleness invariants.The zero-LLM implementation is reproducible across seeds.

A4 AURATown Agent Profiles and Map

AURATown is a fixed five-agent, twenty-location simulation that separates publicly visible scene state from probe-only private state. Its experiments use distinct environment and agent-state tool registries under fixed infrastructure settings.

  • Town and profiles: AURATown uses a 60×60 grid, five named agents, and twenty named locations spanning homes, commerce, civic sites, and parks.Agent identities and starting locations remain fixed across experiments.
  • State visibility: Public state includes location, current action, and nearby agents, whereas availability, emotional_state, unspoken_goal, and beliefs_about_others are probe-only private state.The public/private split supplies the substrate for implicit-intent evaluation.
  • Tool registries: Factual grounding uses an eight-tool registry, while implicit-intent evaluation uses a separate five-tool scripted registry over a fixed scene snapshot.The registries are disjoint by design and target different state-access patterns.
  • Configuration: Experiments use gpt-4o-mini throughout, with probe planning at temperature 0.2 and conversation generation at temperature 0.8.The simulation uses 30-minute ticks, a 6:00–23:00 day cycle, and movement speed 3 grid units/tick.

Comparisons, and Diagrams

The comparisons contrast AURA’s selective, intent-directed probing with literal, reactive, planned, and static-context approaches. The diagrams emphasize its routing and prompt-cost trade-off, while the long-context comparison is explicitly illustrative rather than head-to-head.

  • Motivating contrast: AURA’s motivating contrast adds an implicit availability inference, targeted private-state probing, and a heads-up alert to the literal location answer.On the same scene, vanilla returns only the literal location, whereas AURA reports that Lin Wei is busy.
  • IntentFrame: The IntentFrame emits literal_need, implicit_need, gap, recommended_probes, should_alert, and confidence for downstream Explore and Interact stages.The figure presents one example produced by the LLM-backed IntentInferrer.
  • Long-context comparison: AURA routes selective probing from the inferred gap, while Static-Context reaches FA 0.450 and AURA Full reaches 0.640.Figure 8 sketches prompt-token growth and reports latency and FA from the benchmark tables.
  • Comparison caveat: The long-context diagram is illustrative rather than a head-to-head experiment because Static-Context lacks private state and memory.A true full-state long-context baseline is left for future work.
  • Baselines: The in-suite baselines include Vanilla LLM, Static Context, ReAct, Reflexion, Plan-and-Solve, AURA No Probe, and AURA Full.Generative Agents and ContextAgent are discussed but not rerun as paired baselines.

A9 Strict Precision Rescore for Factual Grounding (Robustness Check)

The strict precision rescore tests whether factual-grounding conclusions survive removal of the completeness component from the original LLM-judge metric. It preserves the architectural headline versus Vanilla, while narrowing the advantage over ReAct and exposing access, hallucination, and latency trade-offs.

  • Strict rescore results: +0.609 strict precision versus Vanilla, p < 10^-4, strengthens the architectural-effect headline.Strict precision excludes the soft completeness component used in the original factual-accuracy score.
  • Strict rescore results: +0.090 strict precision versus ReAct is not significant (p=0.064), while AURA’s hallucination rate is higher: 66.7% versus 51.3%.The combined result indicates only a small architectural contribution above the fair tool-using baseline under strict scoring.
  • Strict rescore results: +0.062 strict precision versus AURA NoProbe is significant (p=0.032), unlike the corresponding lenient contrast (p=0.299).The probing mechanism is more clearly supported under strict scoring, although the overall architectural margin shrinks.
  • Per-category structure: AURA wins strict precision on temporal (0.892) and memory (0.740), but loses to ReAct on social and spatial categories.The original lenient-scoring social probing win does not survive strict scoring.
  • Privacy-sensitive slice: The privacy slice evaluates public-fact queries with forbidden-tool lists, making unnecessary disclosure an explicit outcome alongside factual accuracy.Violations are counted per query-seed cell whenever a forbidden tool fires.
  • Latency qualification: GapRouted is slower than Fixed-Probe at small budgets because IntentInferrer adds an extra LLM call, so probe-count savings do not imply wall-clock savings.Median latency is 4.08 versus 2.37 seconds on factual grounding and 3.97 versus 1.47 seconds on the privacy slice.
  • Latency qualification: The Pareto claim concerns probe count and disclosure rather than wall-clock latency, whose means can be distorted by rare API timeouts.One 604-second timeout raises a mean to 13.85 seconds while the median remains 5.31 seconds.

A12 Prompt Ablation for IntentFrame Calibration

The prompt ablation tests whether IntentFrame gains reflect benchmark memorisation or calibrated gap estimation. Clean few-shot prompting preserves the result, whereas removing examples lowers inferred gaps and makes the comparison non-significant; robustness and benchmark expansion delimit where gains appear.

  • A12 Prompt Ablation for IntentFrame Calibration: The ablation compares leaked, clean, and no-few-shot prompts on the same 25 implicit-intent queries and three seeds.The clean row is the final system; the leaked row is diagnostic only.
  • A12 Prompt Ablation for IntentFrame Calibration: Clean few-shot performance is only 0.037 below the leaked prompt, arguing against memorising names or locations.The no-few-shot prompt instead reduces Intent to 0.677 and makes the Intent-versus-NoIntent gain non-significant.
  • A12 Prompt Ablation for IntentFrame Calibration: Removing examples lowers mean inferred gap from 0.589 to 0.476 and high-gap cells from 43/75 to 22/75.The examples therefore function as calibration for gap-to-budget routing.
  • A13 Cross-Backbone Intent-vs.-NoIntent (Robustness): Three of four additional backbones reproduce the Intent-versus-NoIntent gain, while Gemini’s regression follows JSON-schema failure on 23/25 IntentFrame calls.The Gemini result reflects heuristic fallback after format-compliance failures.
  • A13 Cross-Backbone Intent-vs.-NoIntent (Robustness): Cross-backbone contrasts are +0.16 for gpt-4o-mini, +0.24 for claude-haiku-4-5, +0.25 for qwen-plus, and −0.20 for gemini-2.5-flash.The additional backbone rows are single-seed robustness probes, whereas gpt-4o-mini averages three seeds.
  • A14 Expanded Implicit-Intent v2 Details: The expanded v2 benchmark contains 100 queries across four scene snapshots, five subcategories, and recorded required and forbidden tools.Second-order queries forbid direct private-state access.
  • A14 Expanded Implicit-Intent v2 Details: The benchmark includes availability, mood, appropriateness, latent_goal, and second_order queries, with annotator disagreements concentrated at category boundaries.The pilot achieved Cohen’s κ=0.61 and 68% raw agreement.

A15 Factual-Grounding Per-Category Paired Contrast

Per-category analyses show that proactive probing helps only when passive context leaves relevant environmental uncertainty. Routine grounding and component ablations are largely null because those workloads saturate the metrics, while human evaluations and simulation provide broader but qualified evidence.

  • Factual-grounding per-category contrast: +0.038 aggregate factual accuracy for AURA Full versus NoProbe is near-null, because only the social category improves significantly.The other four categories are saturated by the Perceive/Scene channel.
  • Routine-action grounding: Routine grounding spans only 0.024 GA across five architectures, with every paired test versus Vanilla at p > 0.5.Memory utilisation is approximately 1.0 for every method and many actions are trivially grounded.
  • Routine-action grounding: The routine-grounding null indicates metric saturation on that workload, not failure of the probing mechanism.Factual grounding reaches a 9.1× gap on queries that stress structured-environment access.
  • Component ablation: Component deltas on the routine workload remain small (|∆GA| ≤0.020, |∆FA| ≤0.028), and removing probing saves 1.3 seconds without hurting GA or FA.The same saturation and judge-noise conditions obscure component effects.
  • Component ablation: On factual-grounding chat, the same architecture moves FA from 0.07 for Vanilla to 0.64 for Full, revealing a workload-specific access bottleneck.Routine action grounding does not stress the structured-environment-access channel.
  • Broader evaluation: The 200-step SOTOPIA run scores 7.87/10 overall, with strongest dimensions goal (9.5) and believability (9.0).It produces 44 emergent behaviours across collaboration, routine adaptation, conflict resolution, and group formation.
  • Human evaluation: Human ratings exceed Vanilla on all four dimensions, with Wilcoxon p=0.017 for each dimension and seven of eight raters preferring AURA overall.The rater-aggregated test uses N=8 paired rater means per dimension.
  • Human evaluation: Human-evaluation interpretation is limited by N=8, individual recruitment, incomplete dynamic-state measurement, and moderate inter-rater reliability.The precise magnitude of the effect remains uncertain despite consistent direction.

A18 Heuristic vs. LLM IntentInferrer (Backend Ablation)

The backend ablation separates AURA’s intent-inference contribution from its probe-routing machinery: the LLM backend substantially outperforms the heuristic because it calibrates gaps when surface cues fail.

  • Heuristic backend: The heuristic recovers +0.18 over Literal but remains near Literal on availability and latent_goal.Its trigger vocabulary misses query surfaces such as “where is X?” and “what is X up to?”.
  • Heuristic backend: The heuristic reaches 0.77 on second_order because its trigger vocabulary includes “thinks”, “believes”, and “perspective”.This contrasts with its failures on categories whose implicit needs are not lexically named.
  • Backend comparison: +0.44 aggregate advantage is reported for the calibrated LLM backend over the heuristic backend.The comparison uses the same 25 queries × 3 seeds with identical gap-to-budget mapping, probe loop, and judge.
  • Interpretation: The LLM backend achieves its advantage at 7.3× the heuristic latency, concentrating gains where surface cues fail.The paper identifies LLM-mediated gap calibration, rather than gap-to-budget mapping or probe-loop machinery, as the source of the headline lift.

A19 Cross-Domain Sanity Checks

Cross-domain checks show that AURA’s probing mechanism transfers positively to text-adventure puzzles but not reliably to adversarial games, brittle judged math, or long-context formal reasoning. FANToM likewise provides a negative transfer case where additional probing adds latency without improving accuracy.

  • Puzzle: +5 pp absolute is the cleanest cross-domain gain, with AURA Full reaching 0.70 accuracy versus Vanilla LLM at 0.65 on Puzzle.The comparison covers 20 multi-turn episodes per condition with a single seed.
  • Trust: Trust provides no positive result: AURA Full reaches 1.15 payoff/round versus Vanilla at 1.31 in a six-game-per-condition test.The aggregate is confounded by opponent-mix imbalance, and the authors interpret the result descriptively because opponent private state cannot be probed.
  • Math: Math is reported as null because all conditions reach approximately 1.9% under a brittle judge that misclassifies correct LaTeX-formatted answers.The paper attributes the outcome to repeated parse errors rather than a meaningful condition gap.
  • MemoryArena: MemoryArena is anecdotal: AURA Full reaches 2/5 partial-success while memory-only and no-probe conditions reach 5/5.The authors caution that one paper × five subtasks cannot support a quantitative claim and attribute the direction to probe snippets polluting long mathematical context.
  • FANToM: FANToM shows null accuracy contrasts: Intent versus Literal is ∆acc=+0.003, while Intent versus NoIntent is ∆acc=+0.025.Both contrasts are nonsignificant, and the full narrative context leaves little uncertainty for intent-directed probing to reduce.

A21 LoCoMo Long-Term Conversational Memory (Partial Transfer)

LoCoMo shows partial transfer of AURA’s architecture to long-term conversational memory: the full Intent condition strongly beats Literal, but intent inference adds only a small, nonsignificant increment over NoIntent. The broader evidence also records negative transfer on GAIA, where probing is costly without aggregate benefit.

  • LoCoMo results: +0.151 F1 is the significant Intent-versus-Literal gain on the 200-question LoCoMo split.The paired t statistic is 8.11 with p < 10^-15.
  • LoCoMo results: +0.020 F1 is the additional Intent-versus-NoIntent gain, which is not statistically significant.The paired t statistic is 1.09 with p=0.28.
  • Interpretation: The NoIntent architecture-and-tools layer carries most of the LoCoMo improvement, raising F1 from 0.042 to 0.173.The IntentInferrer contributes only the additional nonsignificant +0.020 F1 on this sample.
  • GAIA negative transfer: GAIA probing degrades Level-1 accuracy by −1.9 pp and spends 22× more wall time with 3.5 tool calls per question.Level-2 conditions tie, yielding no aggregate benefit from probing.
  • GAIA negative transfer: GAIA fails AURA’s scope condition because probes summarize web pages through additional LLM calls rather than reading structurally extractable simulator state.The paper links this setting to compounded hallucination risk and a residual uncertainty that probing is not designed to reduce.

A24 Reproducibility Checklist

The reproducibility materials provide experiment manifests, released code and benchmark assets, documented model and seed handling, and detailed execution protocols. The paper also records deployment safeguards, simulation assumptions, compute usage, and explicit empirical scope boundaries.

  • Code and data: The release includes runners, result JSON manifests, the AURA library, AURATown, environment-grounded queries, and all 100 implicit-intent queries with labels.Each table or figure is traced to a specific result JSON through evaluation/results/MANIFEST.md.
  • Models and APIs: Experiments use gpt-4o-mini for the main agent and judge, with additional vendor APIs for cross-backbone tests.Temperatures are 0.7 for action decisions and 0.1 for judging and intent inference.
  • Determinism: Seed handling varies Python randomness and the OpenAI seed parameter, but stochastic decoding can still prevent strict determinism.The paper characterizes the API seed as best-effort rather than a strict determinism guarantee.
  • Evaluation protocol: Strict-paired factual-grounding evaluation replays identical ground-truth snapshots across conditions with read-only chats.This two-phase protocol preserves the per-tick trajectory for fair comparison.
  • Compute: All experiments run on one CPU machine with remote APIs, and total reported API spend is under $15 USD.The paper reports approximately 8 million input tokens and 200 thousand output tokens.
  • Scope and ethics: The Explore whitelist is the principal misuse mitigation, and deployments over personal-data tools should constrain it and expose inferred needs to users.The paper describes the heads-up prefix in the Interact stage as a minimal version of this transparency measure.
  • Scope and ethics: The paper’s claims are scoped to single-user situated queries with hidden private state and explicitly report negative transfer outside that regime.The stated boundaries include FANToM narrative ToM, GAIA web-grounded factoids, and MemoryArena formal reasoning.
Loading 2606.05557v1…