PaperlayerView exact revision on arXiv

Source-linked AI summary

$\tau$-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions

Rishitosh Kumar Singh, Amir Saeidi, Shamanthak Hegde, Shrinidhi Kumbhar, Chitta Baral

arXiv:2610.19560v1cs.AIcs.CL

TL;DR

Existing agent benchmarks often emphasize task completion without measuring interaction efficiency under emotional variation. τ-Emotion-Bench and Tracer provide an emotion-conditioned benchmark and interaction-aware trajectory generation; experiments find frontier-agent reliability gaps and improvements from Tracer supervision.

  • Problem

    Many agent evaluations and training datasets rely on terminal task success, leaving the efficiency of user interaction and the effect of emotional variation on task-oriented tool use underexplored.

    Source: §1 · p. 2

  • Method

    The paper introduces a ten-domain emotion-conditioned benchmark and Tracer, which generates grounded tasks and training trajectories using a harness where dissatisfied users can terminate inefficient interactions.

    Source: §1 · pp. 2–3

  • Results

    Frontier agents remain unreliable, while Tracer improves Qwen3 models across scales and transfers to unseen domains and external benchmarks.

    Source: §1 · pp. 2–3, §6 · p. 11

  • Takeaways & Limitations

    Emotion-conditioned interactions expose reliability gaps and provide an interaction-aware signal associated with a better balance between tool execution and necessary user follow-up.

    Source: §1 · p. 3, §6 · p. 11

    Tracer-generated data is evaluated only through supervised fine-tuning, and the experiments focus on tool-calling agents rather than settings such as coding or web-based agents.

    Source: LIMITATIONS · p. 11

Abstract

from arXiv · show

In long-horizon tool-calling tasks, LLM agents can fail not only because of tool-use errors, but also because of ineffective interaction with users. Yet existing benchmarks largely assume cooperative and behaviorally stable users, leaving the effect of emotional variation on agent reliability underexplored. We introduce $\tau$-Emotion-Bench, a ten-domain benchmark for evaluating tool-calling agents under controlled emotion-conditioned user interactions while preserving the underlying task objective. To scale both benchmark construction and agent training, we introduce Tracer, a scalable agentic data-generation pipeline that constructs grounded, policy-valid tasks and collects trajectories through an interactive agent--user--environment harness. By allowing dissatisfied users to terminate inefficient interactions, the harness provides an interaction-aware signal that filters trajectories with excessive user dependence and favors effective tool use. Experiments with two user simulators show that frontier agents remain unreliable, with substantial gaps between average task success and consistent task completion. Tracer-generated supervision consistently improves Qwen3 models across scales, generalizes to unseen domains, and transfers to $\tau$-Bench and $\tau$-Trait. Behavioral analyses further show that higher task success is associated with a better balance between tool execution and necessary user follow-up. Overall, $\tau$-Emotion-Bench and Tracer provide a scalable framework for evaluating agent reliability under diverse user behaviors and improving tool-calling agents through interaction-aware supervision.

1 INTRODUCTION

τ-Emotion-Bench evaluates tool-calling agents with emotion-conditioned users while preserving task objectives, and Tracer uses interaction-aware feedback to generate training trajectories. Experiments show persistent reliability gaps for frontier agents and improvements from Tracer supervision across scales and settings.

1 INTRODUCTION

  • Existing evaluation and training often treat terminal task success as the main quality signal, so unnecessary questions and excessive user reliance can go unrewarded as interaction flaws.The motivating gap is measuring whether interaction itself is efficient, not only whether the task is completed.Source: §1 · p. 2
  • τ-Emotion-Bench varies user behavior across five emotion families and fine-grained attributes while preserving the underlying task objective.The benchmark contains 580 executable tasks across ten domains and uses an emotion-conditioned agent–user–environment harness.Source: §1 · p. 2
  • Tracer filters successful but interaction-inefficient trajectories by allowing dissatisfied users to terminate interactions and favoring effective tool use over excessive dependence on users.The harness rewards asking only for information that is genuinely unavailable while relying more on tools and the environment.Source: §1 · p. 2
  • Frontier-agent reliability remains limited: GLM-5.3 reaches 78.6% Avg@3 but 66.5% average Pass^3 with GLM-5.2, while DeepSeek-V4.1-Flash reaches 55.9% Avg@3 but 32.2% average Pass^3 with the Qwen3 user simulator.These gaps indicate substantial room to improve consistent task completion across both evaluation settings.Source: §1 · p. 2
  • Tracer improves Qwen3-1.7B, 4B, and 8B by 17.1–25.2 percentage points in Avg@3, with Tracer-High-8B reaching 54.7%.Tracer-Low, trained on 1,498 trajectories from four domains, generalizes to six unseen domains and achieves the strongest average Pass^3 among compared training methods at all three scales; learned behavior also transfers to τ-Bench and τ-Trait.Source: §1 · p. 3

2 RELATED WORK

Prior benchmarks and training methods increase interaction complexity and scale executable trajectory generation, but controlled emotional variation during task-oriented tool use remains unmeasured. Tracer adds interaction-aware feedback to trajectory collection.

2 RELATED WORK

  • τ-Emotion-Bench varies emotion and behavioral expression while keeping the task and environment fixed, addressing a gap in measuring emotional variation during task-oriented tool use.Related benchmarks cover conversational tool use, policy-constrained interaction, user traits, or affective support, but do not systematically measure this setting.Source: §2 · p. 3
  • Interactive benchmarks progress from conversational tool use and policy-constrained turns to shared environments, ambiguous requests, shifting intent, and varied user traits.The related work includes τ-Bench (Yao et al., 2024), τ 2-Bench, VitaBench, and WildToolBench (Barres et al., 2025; He et al., 2025b; Yu et al., 2026a), as well as τ-Trait and RealUserSim (He et al., 2025a; Zhu et al., 2026a).Source: §2 · p. 3
  • Prior synthetic-environment methods scale executable trajectories, but chiefly assess them through task success, verification, or environment difficulty rather than user-interaction efficiency.Tracer instead uses an emotion-conditioned user harness in which unnecessary user dependence can prompt dissatisfaction and termination.Source: §2 · pp. 3–4

3 τ-EMOTION-BENCH

τ-Emotion-Bench formulates tool use as interaction with both an executable environment and a simulated user, varying user emotion without changing the task objective. Its construction grounds workflows, models emotion-conditioned responses, and verifies tasks before inclusion.

3.1 PROBLEM FORMULATION

  • Each task is a partially observable Markov decision process in which the agent acts through tool calls and natural-language responses and observes tool outputs and user messages.The formulation extends the interaction with an emotion condition while leaving the task instruction, environment, and objective unchanged; emotion is fixed per episode, but its expression evolves with interaction history.Source: §3.1 · p. 4

3.2 TASK CONSTRUCTION

  • Task construction samples valid tool workflows from manually verified dependency graphs, then grounds them in executable environment states and turns them into feasible natural-language requests.A separate feeling generator describes likely user emotion from context and requested actions without accessing the environment database.Source: §3.2 · pp. 4–5

3.3 EMOTION MODELING AND USER SIMULATION

  • The emotion model combines five emotion families and 22 fine-grained emotions with politeness, urgency, and trust attributes to define 594 user conditions.A multi-head classifier and semantic similarity to natural-language anchors select a condition; the simulator then responds to the evolving conversation rather than following a fixed surface style.Source: §3.3 · p. 5

3.4 TASK VERIFICATION AND DATA COMPOSITION

  • All 580 evaluation tasks are manually reviewed under execution, information-sufficiency, and policy criteria, with 101 tasks corrected before inclusion.The benchmark contains 70 tasks each from Airline, Retail, Telecom, and Telehealth, and 50 from each of six other domains; it also includes 1,958 training trajectories across ten domains.Source: §3.4 · p. 5

4 EXPERIMENTS

The experiments evaluate frontier agents, Tracer-trained Qwen3 models, and transfer across domains and benchmarks. Frontier agents show a substantial gap between average success and consistent completion, while Tracer improves performance across model scales and evaluation settings.

4.2 MAIN RESULTS

  • With Qwen3-235B-A22B-Instruct-2507 as user simulator, the best frontier model reaches 55.9% Avg@3 but only 32.2% average Pass^3; under GLM-5.2, GLM-5.3 reaches 78.6% Avg@3 and 66.5% average Pass^3.Results also vary substantially across domains, including low Pass^3 performance on Telecom in both settings.Source: Table 3 · p. 6, §4.2 · p. 7

4.1 EXPERIMENTAL SETUP

  • The evaluation covers 580 tasks across ten executable environments, two emotion-conditioned user simulators, and three trials per task, reporting Avg@3 and Pass^k consistency metrics.The setup also compares Tracer-High, trained on ten domains, with Tracer-Low, trained on four, and existing agent-training baselines for out-of-domain evaluation.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • On six unseen domains, Tracer-Low, trained on 1,498 trajectories from four domains, achieves the highest average Pass^3 among compared methods at 1.7B, 4B, and 8B.At 8B it reaches 8.3% Pass^3, outperforming the named agent-training baselines in average Pass^3.Source: Table 4 · p. 7, §4.2 · p. 8
  • Tracer-High improves Qwen3 models across scales, raising Pass^3 from 1.2% to 10.4% at 1.7B, 3.9% to 17.2% at 4B, and 8.4% to 19.8% at 8B under the Qwen3 simulator.With GLM-5.2, Tracer-High-8B reaches 54.7% Avg@3, exceeding Qwen3-235B-A22B-Instruct-2507 at 51.7%.Source: §4.2 · pp. 7–8
  • Tracer tuning improves Qwen3 models across τ-Bench Airline and Retail and τ-Trait Telehealth and Telecom, including on Pass^2 and Pass^3.These tasks and user simulators differ from those used in Tracer training.Source: Figure 3 · p. 8, §4.2 · p. 9

5 RESULTS ANALYSIS

The analyses examine policy adherence, the balance of tool use and follow-up, empathy, simulator reliability, and training-trajectory patterns. They find that successful interaction is not simply maximized by more tool use or more conversation, and that simulator errors are uncommon in the examined failed trials.

5 RESULTS ANALYSIS

  • Policy violations generally decline with model capability and Tracer tuning, but some stronger-model conditions still reach approximately 67% in Telehealth and 91% in Telecom.Similar behavior from Tracer-Low and Tracer-High, including on unseen domains, suggests a transferred tendency to verify user intent before acting.Source: §5 · pp. 9–10
  • Of 6,961 failed trajectories examined, 124 user turns were confirmed as errors, corresponding to 1.1% of failed trials and at most 1.4% for any emotion family.Most emotion-and-domain combinations contain no hallucinations, though the authors note that annotation uncertainty may remain.Source: Figure 7 · p. 9, §5 · pp. 10–11
  • Tool calls outnumber follow-up actions in 76.2% of Tracer-Low and 78.9% of Tracer-High trajectories, versus 55.6% for AReaL and 22.0% for Simia.The authors describe this pattern as consistent with Tracer’s interaction-aware signal, while noting that the comparison is correlational because dataset construction also differs.Source: Figure 8 · p. 10, §5 · p. 11
  • Higher task success is generally associated with a moderate balance of tool execution and conversational follow-up, while more aggressive tool use does not consistently improve performance.Tracer tuning generally improves performance without shifting toward more tool-heavy interaction.Source: §5 · p. 10
  • Empathetic responses accompany a higher fraction of follow-up questions across model groups, while their association with task success varies by emotion.Empathy is associated with lower success under Enjoyment across groups and under Fear for most groups; effects under Disgust vary substantially.Source: Figure 6 · p. 9, §5 · p. 10

6 CONCLUSION

The paper presents τ-Emotion-Bench as a benchmark for agent reliability under emotion-conditioned interaction and Tracer as a scalable source of training trajectories. Its reported results include broad transfer and a relationship between stronger performance and balanced tool use and follow-up.

6 CONCLUSION

  • τ-Emotion-Bench evaluates tool-calling agents under controlled emotion-conditioned interactions across ten domains.Tracer generates grounded tasks and training trajectories through an emotion-conditioned agent–user–environment harness.Source: §6 · p. 11
  • Tracer-tuned models perform strongly in in-domain and out-of-domain evaluations while using substantially less training data than prior agent-training methods.The conclusion links these gains to a more effective balance between tool execution and necessary user follow-up.Source: §6 · p. 11
  • The findings identify user behavior as an important dimension of agent reliability and report that interaction-aware data generation provides efficient, transferable supervision for tool-calling agents.The harness signal favors efficient task progress rather than task completion alone.Source: §6 · p. 11

LIMITATIONS

Tracer is evaluated only through supervised fine-tuning, and the study focuses on tool-calling agents. Reinforcement-learning supervision and other agent settings remain unexplored.

LIMITATIONS

  • Tracer-generated data is evaluated only as supervised fine-tuning; its effectiveness for reinforcement learning remains unexplored.Source: LIMITATIONS · p. 11
  • The experiments focus on tool-calling agents, leaving coding and web-based agent settings for future work.Source: LIMITATIONS · p. 11

A DATA STATISTICS

The appendix details how tool traces and emotion conditions are constructed, then describes task generation, review, and the final 580-task evaluation benchmark.

A DATA STATISTICS

  • Across ten domains, 1–3 random walks of 2–6 tool calls yielded 42,138 unique traces, with 2,603–8,175 per domain.Evaluation and training tasks used different traces.Source: A DATA STATISTICS · p. 17

A.1 EMOTION MODELING DETAILS

  • The emotion model combines 22 emotions in five families with politeness, urgency, and trust to form 594 conditions and selects simulator instructions using a trained classifier.Ten natural-language instructions per condition produced 5,940 anchors; one instruction is randomly provided to the simulator.Source: §A.1 · p. 17

A.2 EVALUATION DATA

  • From 2,266 candidate tasks, rollout screening retained 884 for review, which included ground-truth corrections, user-accessible identifier swaps, and policy-relevant rewrites.The edits affected 101 tasks for ground-truth corrections and 23 each for identifier swaps and larger rewrites.Source: §A.2 · pp. 17–18
  • The final benchmark contains 580 tasks across ten domains, each paired with an emotion-conditioned instruction and a version without emotion conditioning.Airline, Retail, Telecom, and Telehealth have 70 tasks each; the other six domains have 50 each.Source: §A.2 · p. 18

A.3 TRAINING DATA

Training rollouts are generated from 7,767 tasks, filtered by success and user satisfaction, and sampled into balanced and four-domain training sets.

A.3 TRAINING DATA

  • 7,767 training tasks were rolled out once with Qwen3.8-27B as agent and Qwen3-235B-A22B-Instruct as user simulator, with a 50-step limit.The agent’s reasoning was retained on every assistant turn.Source: §A.3 · p. 18
  • Among unsuccessful rollouts, user-terminated reward −1 trajectories had more Frustration and Exasperation than reward 0 trajectories: 13.7% versus 9.1% and 6.8% versus 3.2%.Excitement was also frequent in both groups: 29.3% for reward −1 and 30.1% for reward 0.Source: §A.3 · p. 18
  • 38.6% of 7,048 retained rollouts succeeded; domain balancing yielded 1,958 trajectories for Tracer-High and 1,498 four-domain trajectories for Tracer-Low.The 7,048 rollouts exclude endpoint-error failures; success required a matching final database state and no user termination with ###UNSATISFIED###.Source: §A.3 · p. 18

A.4 SUCCESSFUL TRAINING TRAJECTORIES

Successful training trajectories generally involve more interaction and tool use than reference sequences, vary by domain, and span multiple emotion conditions in both training sets.

A.4 SUCCESSFUL TRAINING TRAJECTORIES

  • Successful trajectories average 7.2 user turns, 15.7 assistant turns, and 9.5 tool calls, versus 5.2 actions in reference trajectories.Source: §A.4 · p. 18, Table 7 · p. 19
  • Basketball and Movie successful trajectories average 14.7 and 12.0 tool calls, while Airline averages 6.2; Airline and Travel also have among the lowest rollout success rates.Airline and Travel average 889 and 779 reasoning characters per assistant turn, respectively; the passage says their lower success rates suggest successful completion accompanies greater reasoning effort.Source: §A.4 · p. 19
  • Excitement and Peace are the most frequent emotions in both sets, while neither Tracer-High nor Tracer-Low is concentrated in a single emotion.Figure 10 depicts the emotion composition of both sets; their distributions differ because Tracer-Low covers four domains and Tracer-High covers ten.Source: §A.4 · p. 19, Figure 10 · p. 19

B TRAINING DETAILS

The training details specify Qwen3 fine-tuning settings and the evaluation setup for open-weight models. Models use different prompt formats and decoding configurations according to their training and model recommendations.

B.1 FINE-TUNING HYPERPARAMETERS

  • Qwen3 fine-tuning uses the TRL library on four NVIDIA H100 GPUs, with a maximum sequence length of 32,768.The hyperparameters are listed in Table 8.Source: §B.1 · p. 20, Table 8 · p. 20

B.2 OPEN-SOURCE MODELS

  • Models are evaluated through the τ-Bench harness, with prompt format matched to training and decoding settings selected by model and domain.AReaL and Simia-Tau use the τ 2-Bench prompt; most models use model-card sampling settings or greedy decoding in each domain.Source: §B.2 · p. 20, Table 9 · p. 21

C.1 ERROR ANALYSIS

The error analysis finds that failures are concentrated in task execution, though the dominant fault varies by domain. A judge labels errors in failed trajectories and identifies emotional outcomes when users abandon conversations.

C.1 ERROR ANALYSIS

  • A judge reviews failed trajectories and assigns error source, severity, fault tags, turn, and justification; it also labels user abandonment and its dominant reason.The analysis groups fault tags into five categories and fifteen fault types, summarized in Table 11.Source: §C.1 · p. 21, Table 11 · p. 22
  • Missing or extra actions account for 25–36% of faults across domains, incomplete resolution for 20–29%, and wrong tool calls for 14–27%.Misread intent is the most frequent individual fault overall at 15.9%. Policy and interaction faults contribute 4–10% in each domain, while emotional neglect remains below 1.7%.Source: §C.1 · p. 21, Figure 11 · p. 23
  • The dominant fault varies by domain: missing or extra actions lead in eight domains, while wrong tool calls dominate Bank and incomplete resolution dominates Airline.Integrity faults are prominent in Telecom and Basketball, with disclosure before required checks and agent hallucination as cited examples.Source: §C.1 · pp. 22–23

C.2 EFFECT OF TRACER ON AGENT ERRORS

Tracer-High reduces critical agent errors across fault categories, with larger training data generally associated with further gains. Task coverage is broad, but repeated success varies by domain.

C.2 EFFECT OF TRACER ON AGENT ERRORS

  • Tracer-High reduces every fault category at 4B and 8B, with reductions of 10.4–26.2 percentage points at 4B and 3.2–21.6 points at 8B.At 8B, wrong tool calls fall from 38.4% to 20.0% and incomplete resolution from 52.6% to 31.0%; Tracer-Low lies between the base and Tracer-High in nearly every case.Source: §C.2 · pp. 23–24

C.3 TASK COVERAGE

  • Retail has 68 of 70 tasks solved in all three trials, while only 15 tasks are never solved by any evaluated model across three trials.Coverage is high across most domains, but the consistency of successful solutions varies substantially.Source: §C.3 · p. 24, Figure 13 · p. 24

C.4 DETAILED RESULTS

Tracer-High improves Qwen3 models at every scale, and Tracer-Low outperforms compared training methods on average Pass^3. Results also show substantial consistency and domain variation, while the principal findings persist with a second user simulator.

Consistency across trials

  • DeepSeek-V4.1-Flash falls from 55.9% average Pass^1 to 32.2% Pass^3, and most models lose 14–24 points between the metrics.Even the strongest agents solve only about a third of tasks in all three trials.Source: §C.4 · p. 24

Variation across domains

  • Travel and Movie are the hardest domains, with no model exceeding 30.0% Pass^3 on either; GLM-5.3 ranges from 62.0% on Basketball to 2.9% on Telecom.Individual models also show sharp domain strengths, including Muse-Glimmer-30B on School and Kimi-K2.7-Code on Bank.Source: §C.4 · pp. 24–25

Effect of Tracer training

  • Tracer-High raises average Pass^3 over Qwen3 base from 1.1% to 10.4%, 3.9% to 17.2%, and 8.4% to 19.8% at 1.7B, 4B, and 8B.It also improves average Pass^1 at every scale; Tracer-High-8B exceeds Qwen3-235B-A22B-Thinking at 36.0% Pass^1 and 16.5% Pass^3.Source: §C.4 · p. 25

Comparison with agent-training methods

  • Tracer-Low exceeds every compared agent-training method in average Pass^3 at all scales, scoring 7.2%, 13.0%, and 15.2% versus EnvScaler’s 3.1%, 9.6%, and 11.8%.On the six domains excluded from its training, Tracer-Low exceeds EnvScaler in average Pass^3 at all three scales.Source: §C.4 · p. 25

Robustness to the user simulator

  • With GLM-5.2, absolute scores rise while the top four models remain largely the same, and Tracer-High retains gains over the base models at each scale.Its average Pass^1 gains are 25.2, 20.2, and 17.5 points, and average Pass^3 gains are 15.4, 18.0, and 14.9 points at 1.7B, 4B, and 8B.Source: §C.4 · p. 25

D.1 TOOL DEPEDENCY GRAPHS

This appendix section documents tool-dependency graphs and domain-specific policies and prompts that scope supported agent tasks and specify tool-use constraints, required checks and confirmations, and operational safeguards. It also describes simulator prompts and synthetic domain databases used to ground valid task arguments.

D.1 TOOL DEPEDENCY GRAPHS

  • Across domains, tool graphs contain 24–99 valid transitions at 0.25–0.43 density, with 34–57% of connected pairs valid in both directions.Graphs range from RESTAURANT’s 9 tools to BANK’s 17; root-eligible starting tools include write tools such as cancel_reservation or freeze_card.Source: §D.1 · pp. 25–26, Figure 14 · p. 27

D.2 DOMAIN POLICIES

  • Each domain’s policy is reproduced verbatim and supplied to the agent as its system prompt.This establishes the policies as task instructions for the agent.Source: §D.2 · p. 29

D.2.1 AIRLINE POLICY

  • The airline policy requires explicit confirmation before database updates, one tool call at a time, and denial of requests that violate policy.It also prohibits unsupported information or subjective recommendations and permits human transfer only when the request cannot be handled under policy.Source: §D.2.1 · p. 29

D.2.2 RETAIL POLICY

  • The retail policy requires identity authentication at conversation start and explicit confirmation before consequential database actions.It also limits the agent to one tool call at a time, one user per conversation, and actions allowed by order status and policy.Source: §D.2.2 · pp. 30–31
  • Delivered-order returns and exchanges require a status check and no conflicting request; exchanges replace each eligible unit with an available option of the same product.Return and exchange requests are tracked per unit, so an unaffected identical item may still be acted on.Source: §D.2.2 · p. 32
  • Retail returns require confirmation of the order, items, and payment method, with refunds sent to the original payment method or an existing gift card.After confirmation, the order status is updated and the user receives return instructions by email.Source: §D.2.2 · p. 32
  • Retail exchanges require a payment method for any price difference, sufficient gift-card balance when used, and user confirmation; they update the status without placing a new order.After confirmation, the user receives an email explaining how to return the items.Source: §D.2.2 · p. 32

D.2.3 TELECOM POLICY

  • Before telecom database changes, agents must list actions and avoid simultaneous tool calls and user responses; they must deny policy-violating requests and relay tool output verbatim.The formatting policy permits only removal of unnecessary formatting from returned tool output.Source: §D.2.3 · pp. 32–33

D.2.4 TELEHEALTH POLICY

  • Telehealth agents must authenticate patients before sharing sensitive information, obtain explicit confirmation before consequential updates, and refer medical questions to providers rather than give advice.Authentication uses an exact-match email address or full name plus date of birth.Source: §D.2.4 · pp. 33–34

D.2.5 SCHOOL POLICY

  • School agents must list action details and obtain explicit user confirmation before updating the registration database.This applies to registration changes, grading options, health insurance, and payment or refund records.Source: §D.2.5 · p. 35
  • Initial school registration requires student identification and task details, followed by explicit confirmation of the proposed registration and payment allocation before submission.The details include term, study level, courses and grading options, insurance choice, total due, and payment allocation.Source: §D.2.5 · pp. 35–36
  • School registration changes must preserve the existing term and study level, respect course capacity and schedule constraints, and handle tuition increases or decreases through charges or refunds.Before an update, the agent must present the planned changes and obtain explicit confirmation.Source: §D.2.5 · p. 36

D.2.6 BASKETBALL POLICY

  • Basketball agents must authenticate the player, protect other players’ personal data, and obtain confirmation before supported write actions.Writes include address changes, valid game-status updates, and scoring events subject to game and event prerequisites.Source: §D.2.6 · pp. 37–38

D.2.7 RESTAURANT POLICY

  • The restaurant policy passage identifies patron authentication and locating the patron account, plus access to the patron’s own profile information and tickets, as supported tasks.The supplied passage ends after listing tickets, so it does not establish the full set of restaurant capabilities.Source: §D.2.7 · p. 38
  • Restaurant tools support ticket cancellation, eligible ticket changes, and tips, with saved payment instruments and action-specific constraints governing updates.Examples include changing delivery dropoff or dine-in table information, replacing plates only with available plates of the same dish, and requiring confirmation before changes.Source: §D.2.7 · pp. 38, 40
  • Restaurant agents must authenticate patrons before accessing their data, serve only one patron per conversation, and obtain explicit confirmation before database updates.Only placed tickets can generally be modified or cancelled; cancellation also requires the ticket reference and a reason, while tips are allowed on uncancelled tickets.Source: §D.2.7 · pp. 38–40

D.2.8 TRAVEL POLICY

  • Travel agents may modify only bookings in the authenticated traveler's profile; permitted changes include departure dates, add-ons, traveler details, rooming, and agent meetings.A new departure must have sufficient availability, and price differences for departure or add-on changes are charged or refunded using a saved payment method.Source: §D.2.8 · pp. 41–42
  • Travel agents may book packages only for available departures with enough slots, after collecting traveler, rooming, add-on, insurance, and saved-payment details and confirming the booking summary.Payments must total the computed price, and all travelers in a booking share the same package and departure date.Source: §D.2.8 · pp. 41–42
  • Travel actions require the relevant traveler and booking details, package-policy checks before cancellation, and explicit user confirmation before writes.Unsupported changes or unclear package policies require transfer to a human agent.Source: §D.2.8 · pp. 42–43

D.2.9 MOVIE POLICY

  • Movie workflows require checking show and seat availability, previewing exact prices, and obtaining confirmation before booking or updating a booking.Booking changes are limited to seats; cancellation requires an eligible show status and confirmation, while unsupported requests are transferred to a human agent.Source: §D.2.9 · pp. 44–45

D.2.10 BANK POLICY

  • Bank workflows begin by authenticating through the user’s email, restrict assistance to that authenticated user’s data, and allow at most one tool call at a time.The policy also requires listing the action before a database write.Source: §D.2.10 · p. 46
  • Bank actions require authentication, ownership checks, READ-tool validation before WRITEs, and explicit user confirmation; unsupported or constraint-violating requests must be denied.Source: §D.2.10 · p. 48

D.3.1 AGENT SYSTEM PROMPT

AGENT SYSTEM PROMPT

  • The agent prompt frames the role as customer service under the supplied policy and requires valid JSON output.Source: AGENT SYSTEM PROMPT · p. 48

D.3.2 USER SIMULATOR SYSTEM PROMPT

USER SIMULATOR SYSTEM PROMPT

  • When given an emotional profile, the simulator should reflect it in tone, patience, cooperation, and information sharing, and may end early with ###UNSATISFIED### when interaction needs are ignored or friction occurs.Source: USER SIMULATOR SYSTEM PROMPT · p. 49

D.4 CORE ENTITIES

  • Each domain uses an independently generated synthetic database whose entity counts define valid arguments, such as reservation or product IDs, for grounded task generation.The evaluation-set entity counts are summarized in Table 15.Source: §D.4 · p. 49, Table 15 · p. 49

E TASK TRAJECTORIES

This appendix presents human-trial task trajectories across airline, retail, telecom, and telehealth, pairing emotional user instructions and expected actions with annotated multi-turn conversations, tool calls, and results. A telehealth example follows an agent rescheduling and booking visits and confirming a copay at the user’s request.

E TASK TRAJECTORIES

  • Each trajectory pairs a task instruction with emotional context and expected actions, then records the multi-turn user–assistant conversation with tool calls and results.The role labels are [U] user, [HA] human assistant, [T] tool call, and [R] tool result.Source: E TASK TRAJECTORIES · p. 49

E.1.1 AIRLINE

  • The airline example books the requested direct evening business-class flight HAT277 for $270 after collecting passenger, baggage, and insurance details.The reservation is confirmed as HATHAT; the user then asks the assistant to charge the card ending in 2008 and send the e-ticket.Source: §E.1.1 · pp. 50–51

E.1.2 RETAIL

  • The retail example changes a pending order’s payment method and shipping and default addresses, then lists available Action Camera variants for a possible purchase.The user accepts the available 4K waterproof black variant and asks whether it is rugged for outdoor use.Source: §E.1.2 · pp. 53–54

E.1.3 TELECOM

  • The telecom example adds an iPhone 14 to the customer account, creates medium-priority ticket TICKET241, and responds to a request to verify device specifications.The supplied trajectory ends with the assistant affirming that the iPhone 14 was added successfully.Source: §E.1.3 · pp. 55–56

E.1.4 TELEHEALTH

  • The telehealth example begins with a request to schedule a next-day checkup with Dr. Margaret Thompson and reschedule an existing appointment with Dr. Robert Smith.After obtaining the patient’s email, the assistant retrieves an existing 10:00 appointment on 2025-12-17 with Dr. Smith; no rescheduling outcome is shown in this part.Source: §E.1.4 · p. 57
  • The agent rescheduled Dr. Smith’s appointment to December 17 at 4:00 PM and scheduled Dr. Thompson for December 18 at 2:00 PM.Both appointment tools returned success, and the assistant summarized the two updated appointments.Source: §E.1.4 · pp. 57–58
  • After the user asked, the assistant confirmed Dr. Thompson’s December 18 visit had a $50.00 copay.The user then expressed satisfaction and ended the interaction.Source: §E.1.4 · p. 58

Related papers

Loading 2610.19560v1…