PaperlayerView exact revision on arXiv

Source-linked AI summary

Intent-Driven Situation Tracking for User-Centric Multi-Turn Agents

Meiling Tao, Yiling Tao, Peng Wang

arXiv:2608.15755v1cs.AI

TL;DR

User-centric multi-turn agents must track evolving intents, grounded facts, missing information, and constraints, but existing methods rarely represent this situation explicitly. IDSS maintains fact and task-state layers linked by constraint propagation, and across three benchmarks and eight LLMs improves task completion, preference elicitation, and interaction efficiency, especially for multi-entity coordination and evolving constraints.

  • Problem

    User-centric multi-turn agents must reconstruct changing goals, facts, missing variables, and constraints from dialogue histories because existing context management rarely separates grounded facts from task-state judgments.

  • Method

    IDSS is a training-free framework that maintains explicit fact and state layers and propagates newly observed facts to intents, variables, constraints, and execution status.

  • Results

    Across three interactive benchmarks and eight LLMs, IDSS improves task completion, preference elicitation, and interaction efficiency, especially on multi-entity coordination and evolving-constraint tasks.

  • Takeaways & Limitations

    Explicit, decision-oriented situation states offer an effective alternative to relying only on past interaction history for user-centric multi-turn agents.

  • Takeaways & Limitations

    IDSS is designed for tool returns with textual fields convertible into structured facts, and its experiments use simulated users rather than real users.

Abstract

from arXiv · show

User-centric multi-turn agents must act on an evolving task situation shaped by changing user intents, accumulated tool-grounded facts, missing information, and execution constraints. Existing context-management methods improve the use of past interaction history, but rarely maintain an explicit situation state that separates grounded facts from task-state judgments. As a result, agents often need to infer fine-grained attributes, task dependencies, and constraint satisfaction implicitly from dialogue traces. We propose Intent-Driven Situation States (IDSS), a training-free framework that maintains an explicit situation state alongside the dialogue. IDSS parses tool returns into provenance-aware entities and attributes, tracks user intents, required variables, constraints, and execution status, and propagates new facts to task constraints to update action executability. This allows agents to avoid infeasible actions, advance dependent goals, and reuse relevant information without repeatedly searching raw history. Experiments on three interactive benchmarks across eight LLMs show that IDSS improves task completion, preference elicitation, and interaction efficiency, with clear gains on tasks involving multi-entity coordination, evolving user constraints, and constraint-aware replanning. Ablations and error analyses show that these improvements come from the interaction between fact persistence, intent-centered state tracking, and constraint modeling. These results suggest that explicit situation tracking offers an effective alternative to history-centric context management for reliable user-centric multi-turn agents.

1 Introduction

User-centric multi-turn agents must maintain an evolving task situation rather than rely only on dialogue history. IDSS addresses this need with explicit fact and task-state layers, and improves outcomes across benchmarks and models.

  • Motivation: Evolving goals, preferences, tool facts, and constraints require agents to track grounded facts, active intents, missing variables, and task progress.Without explicit tracking, agents may omit facts, drift from intents, execute prematurely, or violate constraints.
  • Research gap: Existing memory and compression methods improve information access but leave agents to infer the current task situation from retrieved or compressed history.Grounded facts are not necessarily separated from task-state judgments such as active intents, missing variables, and constraint satisfaction.
  • Approach: IDSS maintains a fact layer and a state layer without additional LLM calls, tracking tool-grounded entities, intents, variables, constraints, and execution status.Deterministic parsing updates facts incrementally, while the state layer is maintained during action generation.
  • Approach: Cross-layer constraint propagation uses newly observed facts to block infeasible intents, activate subsequent goals, and reuse relevant information without repeatedly searching raw history.The framework combines explicit situation tracking with compact residual dialogue history in the prompt.
  • Results: Across three interactive benchmarks and eight LLMs, IDSS improves task completion, preference elicitation, and interaction efficiency, especially for multi-entity coordination and evolving constraints.The evaluation covers τ-bench, VitaBench, and UserBench.

2 Related Work

Related work improves retention, compression, training, or structured interaction, but existing approaches generally do not maintain IDSS's explicit separation of grounded facts from task-state judgments and constraint dependencies.

  • Interactive agents: Benchmarks expose persistent challenges in information retention, intent tracking, and holistic planning as multi-turn conversations grow longer.τ-bench emphasizes policy-following and tool-use consistency, VitaBench shifting intents and temporal-spatial reasoning, and UserBench incremental preference elicitation.
  • Context management: External memory and in-context compression reduce context pressure, but do not necessarily maintain an explicit current task situation.These methods include working-memory management, fact consolidation, temporal knowledge graphs, reflection-based retrieval, and dialogue restructuring.
  • Dialogue state tracking: Dialogue state tracking uses slot-value pairs within predefined ontologies, making it unsuited to open-ended entities, dynamic constraints, and multi-intent dependencies in tool-augmented settings.Recent extensions add LLM function calling or knowledge-graph reasoning, but retain the broader ontology-centered framing.
  • Agent state tracking: StateAct and PABU track interaction state but respectively omit explicit fact-goal separation or constraint dependencies, while PABU also requires fine-tuning and assumes linear progress.IDSS instead links grounded facts and task-state judgments through constraint propagation.

3 Method

IDSS replaces history-only decision-making with an explicit situation state that combines structured facts, user-task progress, and compact residual dialogue. Its fact, intent, variable, and constraint tracking supports executable-action selection, constraint-aware replanning, and efficient task completion.

  • Situation-Augmented Policy: IDSS replaces the history-only policy with a situation-augmented policy using structured situation state and compact residual history.The residual history retains recent turns while replacing parsed tool outputs with fact-layer references.
  • Situation State: The situation state combines a fact layer for tool-grounded entities and attributes with a state layer for intents, variables, constraints, and execution status.Cross-layer propagation aligns task progress with newly observed facts.
  • Fact Layer: The fact layer incrementally stores interrelated entities and attributes, supports cross-turn querying, and resolves conflicts by source credibility and recency.Operations include ADD, UPDATE, DELETE, and MERGE; tool-observed attributes take precedence over agent-inferred ones.
  • State Layer: The state layer tracks intent status, dependencies, required variables, and constraints to determine whether intents are executable or which question, retrieval, or tool action comes next.Variables are marked askable, retrievable, or derivable, while intents may be active, pending, completed, or blocked.
  • Constraint Propagation: Cross-layer propagation reevaluates grounded constraints after fact updates, blocks violated intents, and can activate dependent alternatives instead of retrying infeasible actions.A fare-class conflict can block flight modification and activate cancellation-and-rebooking while preserving the original requirements.

4 Experiments

Across three interactive benchmarks and eight LLMs, IDSS delivers strong task, preference, and efficiency results, with clearest gains on coordination and evolving-constraint tasks.

  • Main Results: IDSS delivers the strongest overall performance among compared methods and improves consistently over ReAct across three benchmarks and eight LLMs.Its normalized profile also shows balanced gains across evaluation axes rather than improvement on only one benchmark or metric.
  • τ-bench: 1.5 Avg@4 and 2.0 pass4 points are IDSS’s gains over U-Fold on τ-bench Airline.Airline involves cross-turn dependencies and blocking conditions from rebooking, cancellation, baggage, and fare-class restrictions.
  • τ-bench: 0.6 Avg@4 and 1.8 pass4 points are IDSS’s positive gains over U-Fold on τ-bench Retail.These gains are smaller than on Airline.
  • VitaBench: 1.7 and 2.1 points are IDSS’s gains over U-Fold on VitaBench OTA and Cross-domain tasks.These settings require multi-entity coordination, cross-service composition, and state inheritance.
  • UserBench: Approximately 4.6% on Score and 4.0% on CER are IDSS’s relative improvements over U-Fold on UserBench.IDSS achieves the best Score and CER across evaluated LLMs; preference-elicitation gains are smaller and vary across backbones.
  • Ablation Study: Fact and state layers make complementary contributions: removing the fact layer reduces τ-bench Avg@4 by 2.2 points and pass4 by 3.5 points, while removing the state layer drops UserBench Score by 8.4%.On VitaBench, removing the fact and state layers causes drops of 2.5 and 2.8 points, respectively; constraint modeling provides smaller but consistent gains.
  • Error Analysis: Fact omission decreases from 14% to 8% and missing-information errors from 8% to 5% with IDSS.Constraint-violation rates change modestly across methods, ranging from 5% to 8%.
  • Efficiency Analysis: IDSS achieves 50.5% average performance with 7.8 interaction turns, the fewest turns, and zero extra LLM calls.It renders the current situation state with compressed history while preserving reusable facts and blocking infeasible or underspecified actions.

5 Conclusion

IDSS explicitly tracks the current task situation by separating tool-grounded facts from evolving task-state judgments. Across three interactive benchmarks and eight LLMs, it improves task completion, preference elicitation, and interaction efficiency, especially for multi-entity coordination and evolving constraints.

  • IDSS maintains an explicit situation state that separates tool-grounded facts from intents, missing variables, constraints, and execution status.
  • IDSS improves task completion, preference elicitation, and interaction efficiency across three interactive benchmarks and eight LLMs.
  • The clearest gains occur on tasks involving multi-entity coordination and evolving constraints.
  • The results support explicit, decision-oriented situation states over relying only on access to past interaction history.

Limitations

IDSS is designed for interactive tasks whose tool returns expose textual fields that can be converted into structured facts. Extending the framework to richer outputs, tool environments, and multimodal observations remains future work, and real-user studies are also needed.

  • IDSS currently targets interactive tasks where tool returns expose textual fields convertible into structured facts.
  • The paper identifies richer output formats, diverse tool environments, and multimodal observations as directions for extending IDSS.
  • The experiments use simulated users for reproducibility and controlled comparison across models rather than real users.
  • Studies with real users could further validate support for naturally expressed goals, corrections, and preferences.

Ethical Considerations

The work evaluates training-free context management in public or simulated benchmark environments without collecting personal data or involving human participants. It cautions that IDSS does not remove risks from incorrect tool outputs, hallucinations, or inappropriate high-stakes deployment.

  • The study uses public or simulated benchmark environments for training-free context-management experiments.
  • The authors report no personal-data collection and no experiments with human participants.
  • IDSS does not eliminate risks from incorrect tool outputs, model hallucinations, or inappropriate deployment in high-stakes settings.
  • High-stakes deployments should include task-specific safeguards, logging, and human oversight when actions may affect users.

A.1 Benchmarks

The benchmarks evaluate multi-turn agents across customer-service, life-service, and incrementally specified user-interaction settings. The supplied example illustrates how situation state records entities, variables, constraints, intents, and dependencies for constraint-aware replanning.

  • τ-bench simulates 3 to 8-turn dialogues in Retail and Airline domains with API tools, rules, and follow-up user requests.
  • VitaBench covers Delivery, In-store, OTA, and Cross-domain scenarios with verbose tool outputs, multi-entity coordination, and progressively revealed constraints.
  • UserBench evaluates underspecified interactions requiring proactive clarification and preference elicitation, reporting Score, CER, and PE metrics.
  • The example state records tool-grounded values, askable or retrievable variables, constraint statuses, active intents, and dependencies between goals.
  • The example shows a blocked flight modification activating cancellation and rebooking, with checked-bag addition pending completion of that dependent intent.

C.1 τ-bench Retail: Multi-Item Exchange with Conditional Preferences

The exchange task requires coordinating two items while honoring a conditional keyboard fallback. IDSS preserves tool-grounded variants, tracks blocked and active intents, and obtains confirmation before executing both exchanges.

  • The user requests exchanges for a mechanical keyboard and smart thermostat with specific compatibility, size, backlight, and fallback preferences.
  • IDSS records tool-grounded product availability, prices, payment information, and the keyboard intent’s dependency on the unavailable primary variant.
  • IDSS activates the no-backlight keyboard fallback after detecting the preferred variant is unavailable, while keeping the thermostat exchange active.
  • The agent recommends the available keyboard and thermostat variants, explains the fallback, and asks for confirmation before proceeding.
  • ReAct may retry an infeasible keyboard exchange, lose the fallback preference, select the wrong variant, or exchange only one item, requiring 2 to 3 correction turns.

C.2 VitaBench Cross-domain: CS Trip with Schedule Conflict and Delivery Constraint

The task requires repairing a CS trip after a meeting changes the departure time, while coordinating train arrival, travel time, reservation timing, and same-day sunscreen delivery. IDSS propagates these constraints and requests confirmation before booking.

  • A Saturday meeting requires a later train and cascading changes to transportation, the CS reservation, friend notification, and delivery planning.
  • IDSS tracks candidate trains, the paid CS reservation, the after-13:00 departure constraint, travel time, and unresolved reservation timing.
  • The initial G661 option arrives at 14:15, making a 14:00 CS start infeasible after 1.5 hours of travel; the new CS time remains unknown.
  • The agent recommends G661 and proposes moving the CS reservation to 15:30, noting that G515 would push the activity to 16:18.
  • IDSS finds sunscreen available for delivery today from 20:00 to 22:00, but asks whether to add a quiet-delivery instruction before ordering.
  • ReAct may change the train without updating the CS reservation or verifying delivery timing, leaving timing inconsistencies after 3 additional correction turns.

C.3 UserBench: Business Trip with Underspecified Preferences

The business-trip task begins with underspecified baggage and restaurant-quality preferences. IDSS elicits these preferences, records them as constraints, and filters flight and restaurant recommendations accordingly.

  • The user requests a Los Angeles–NYC flight and two business-dinner restaurants but initially leaves baggage and rating thresholds underspecified.
  • The resulting preferences require carry-on allowance, checked-bag service, business dining, and no 1- or 2-star reviews.
  • The agent asks about carry-on and checked baggage needs, restaurant ambiance, and tolerance for lower ratings, eliciting the relevant preferences.
  • IDSS marks flight search active, restaurant search dependent on preference elicitation, and the rating threshold as an unknown variable.
  • IDSS recommends Delta DL456 at $350 plus a $50 checked-bag fee and restaurants whose reviews are at least 3-star with a Business Dining tag.
  • ReAct may choose a cheaper flight without checked-bag service or recommend Urban Grill despite its lower-rated reviews, producing misaligned recommendations and low preference elicitation.
Loading 2608.15755v1…