Source-linked AI summary
TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios
Yuanzhe Shen, Zisu Huang, Zhengyuan Wang, Muzhao Tian, Zhengkang Guo, Chenyang Zhang, Shuaiyu Zhou, Zengjie Hu, Dailin Li, Jingwen Xu, Kaimin Wang, Wenhao Liu, Tianlong Li, Fengpeng Yue, Feng Hong, Cao Liu, Ke Zeng
TL;DR
Existing benchmarks insufficiently capture global constraints, coordinated tool use, and evolving behavior in long multi-turn tasks. The paper introduces TRIP-Bench, a realistic travel-planning benchmark, and GTPO, an online multi-turn RL method; experiments show strong models struggle while GTPO yields consistent gains over baselines.
Problem
Existing benchmarks underrepresent global constraint adherence, long-horizon planning, coordinated tool use, and dynamic multi-turn user behavior.
Method
TRIP-Bench provides a travel-planning benchmark with 18 tools, diverse constraints and interactions, while GTPO applies online multi-turn RL with specialized reward normalization and differencing.
Results
Strong models struggle on TRIP-Bench, while GTPO-trained Qwen2.5-32B-Instruct delivers consistent gains over strong baselines and outperforms Gemini-3-Pro.
Takeaways & Limitations
TRIP-Bench exposes substantial gaps in long-horizon interactive agents, and GTPO provides an online training approach for improving robust interaction.
Abstract
from arXiv · showhide
As LLM-based agents are deployed in increasingly complex real-world settings, existing benchmarks underrepresent key challenges such as enforcing global constraints, coordinating multi-tool reasoning, and adapting to evolving user behavior over long, multi-turn interactions. To bridge this gap, we introduce \textbf{TRIP-Bench}, a long-horizon benchmark grounded in realistic travel-planning scenarios. TRIP-Bench leverages real-world data, offers 18 curated tools and 40+ travel requirements, and supports automated evaluation. It includes splits of varying difficulty; the hard split emphasizes long and ambiguous interactions, style shifts, feasibility changes, and iterative version revision. Dialogues span up to 15 user turns, can involve 150+ tool calls, and may exceed 200k tokens of context. Experiments show that even advanced models achieve at most 50\% success on the easy split, with performance dropping below 10\% on hard subsets. We further propose \textbf{GTPO}, an online multi-turn reinforcement learning method with specialized reward normalization and reward differencing. Applied to Qwen2.5-32B-Instruct, GTPO improves constraint satisfaction and interaction robustness, outperforming Gemini-3-Pro in our evaluation. We expect TRIP-Bench to advance practical long-horizon interactive agents, and GTPO to provide an effective online RL recipe for robust long-horizon training.
1. Introduction
TRIP-Bench addresses limitations in existing benchmarks by evaluating long-horizon planning, complex constraints, tool coordination, and dynamic multi-turn interaction. It combines a realistic travel-planning benchmark with GTPO, an online multi-turn RL method that improves robust constraint adherence and preference adaptation.
- Motivation: Existing benchmarks underrepresent global constraints, long-horizon planning, iterative refinement, and evolving user behavior.Many tasks are single-turn or use short tool-call chains, while revisions, rollbacks, and version control remain under-modeled.
- TRIP-Bench: TRIP-Bench evaluates long-horizon Tasks, complex Rules, diverse multi-turn Interactions, and reasoning-driven Planning.The benchmark is designed around robust instruction following under global constraints and coordinated tool use.
- TRIP-Bench: 15 turns, 150+ tool calls, and 200k+ tokens define the hardest TRIP-Bench cases.The hard split includes long dialogs, feasibility transitions, ambiguous intent, style shifts, and version control; most models score below 10% in strict mode, while GPT-5.2 reaches 45% in loose mode.
- GTPO: GTPO combines turn-level reward normalization, global-instruction normalization, and turn-level reward differencing for online multi-turn RL.The method targets stable adherence to global rules and adaptation to dynamic preferences.
- Results: GTPO-trained Qwen2.5-32B-Instruct outperforms the SFT model by over 10 pp loose and over 5 pp strict, and exceeds the base model by over 20 pp loose.The paper also reports that the trained model outperforms Gemini-3-Pro on TRIP-Bench.
- Implications: The benchmark and experiments expose substantial limitations in existing models under long-horizon reasoning, diverse user behavior, and global constraint adherence.The authors position the environment as enabling scalable benchmarking and multi-turn training and evaluation under complex constraints.
2. Related Work
Prior agent benchmarks have progressed from single-turn tool use toward multi-turn interaction, but often limit autonomy, behavioral diversity, or long-horizon reasoning. TRIP-Bench is presented as a unified evaluation of complex instruction following, long-horizon reasoning, and diverse user–agent behavior.
- Agent-user interaction benchmarks: Early benchmarks focus on single-turn, single-step tool use, while later benchmarks add single-turn multi-step tasks and conversational tool execution.Multi-turn settings such as ToolTalk and BFCL support dialogue but rely on predefined trajectories that constrain autonomy and behavioral diversity.
- TRIP-Bench: TRIP-Bench simultaneously stresses complex instruction following, long-horizon reasoning, and diverse user–agent interaction behaviors.It addresses the gap with long-horizon tasks involving complex rules, rich multi-turn interactions, and spatiotemporal planning.
- Multi-turn Reinforcement Learning: Multi-turn reinforcement learning research commonly treats the user query as fixed context and optimizes primarily for a single response.This leaves multi-turn tool-use settings less represented than single-turn multi-step tasks.
3. TRIP-Bench
TRIP-Bench constructs feasible, difficulty-controlled travel tasks with rich constraints, progressive revisions, and dynamic user behaviors, then evaluates agents with aggregate strict and loose metrics. Its training pipeline also uses repaired trajectories and GTPO-specific multi-turn reward processing.
- Data construction: The expanded dataset covers 40 cities, 6k+ attractions, 80k+ hotels, 400k+ restaurants, and 1M+ distinct products.POI attributes are enriched and formatting, logic, and consistency issues are repaired to support reliable generation and evaluation.
- Tool environment: A unified interface provides 18 tools for transportation, attractions, restaurants, hotels, and general utilities.Field-based filtering, sorting, and result-size control support systematic evaluation of tool invocation and compositional decisions.
- Task synthesis: Rubric construction pairs each travel requirement with a generator producing selection ranges and feasible IDs and a validator checking individual IDs.The rubric contains approximately 40 requirement categories and over 80 natural-language expressions.
- Task synthesis: Modification chains contain up to three progressively restrictive steps that model iterative user refinement.Trajectory-based trimming and rubric-level adjustment reduce redundant chains whose earlier constraints entail later ones.
- Task generation: Tasks are partitioned into easy, mid, and hard tiers using trip length, city count, constraint count, and simulated user behavior difficulty.Feasibility is verified by retaining candidates satisfying every validator and requiring |C| ≥ (4–10)× trip length in days.
- Behavioral scenarios: Hard subsets model long interactions, feasibility–infeasibility transitions, ambiguous intent shifts, and plan merging or redirection.LIT extends dialogue length, FIT requires rollbacks, AIS reveals preferences through interaction, and PMR switches, rolls back, or merges similar itineraries.
- Validation: Two-stage validation combines model scoring and manual review with full-plan checks because local feasibility does not ensure global feasibility.The process filters unrealistic preference–budget combinations and checks whether flagged spatiotemporal issues are repairable.
- Evaluation metrics: Evaluation reports OverallStrict with zero violations and OverallLoose with strict feasibility plus up to two soundness and one user-constraint violation.The metrics aggregate violations across feasibility, planning soundness, and user constraints.
4. Method
GTPO trains an online multi-turn policy using rollout-based rewards, global and turn-level normalization, and turn-wise reward differencing. Its objective optimizes clipped policy updates while regularizing divergence from a reference policy.
- Rollout Sampling: GTPO samples groups of multi-turn rollouts from the current policy for a given dialogue context.Each rollout consists of user inputs and assistant responses across its completed turns.
- Reward Construction: At each turn, binary constraint scores are combined into a raw reward over the constraints applicable at that turn.The reward construction uses turn-specific constraint indicators and scores.
- Reward Normalization: Global instruction normalization applies z-score normalization across the turns where each constraint applies before aggregating the normalized turn reward.This treats each constraint over its own applicable-turn sequence.
- Reward Differencing: Turn-wise reward differencing emphasizes relative improvement because later-turn rewards can inherit structure from earlier turns.The method is designed to distinguish progress at the current turn from reward attributable to prior turns.
- Turn-Level Processing: Turn-level normalization uses only turns with at least half the rollout group evaluable, masks context-budget-overflow turns, and keeps advantages turn-local.The resulting turn-level advantages do not propagate across turns.
- Final Objective: The GTPO objective maximizes a clipped PPO-style importance-weighted advantage while penalizing per-token KL divergence from a reference policy.The objective uses threshold ϵ for clipping and coefficient β for the KL penalty.
5. Experiments
Experiments evaluate recent models under explicit and default reasoning configurations, finding severe difficulty on TRIP-Bench and consistent reasoning gains that remain insufficient on hard cases. GTPO outperforms SFT and GRPO across evaluation settings and scales to stronger models.
- Experimental Setup: Models are evaluated under explicit-reasoning-disabled and default-reasoning configurations.
- Experimental Setup: The trained models are evaluated only on easy and mid subsets because hard cases can exceed 128k-token context limits.
- Main Results: 18.5 is the best strict score on Easy, while the highest loose score reaches only 45.
- Main Results: 26 percentage points: enabling thinking raises Easy-strict performance from 5.0 to 31.0.
- Main Results: Thinking improves easier splits and often raises Hard-loose scores by more than 15 points, but Hard-strict performance remains uniformly low.
- GTPO: GTPO consistently outperforms SFT and GRPO, reaching 40 on Mid-loose and 21 on Easy-strict with Qwen2.5-32B-Instruct.
6. In-Depth Analysis
The analysis examines resource-performance trade-offs, constraint failures, interaction-mode differences, training ablations, simulator reliability, and exploration reliability. It shows that deeper reasoning helps but remains costly, while global consistency and single-try reliability remain difficult.
- Resource Use: Performance scales approximately linearly with output tokens, while performance gains versus inference cost are closer to logarithmic because multi-step reasoning adds input-token overhead.
- Constraint Analysis: Figure 4 breaks down the top-15 highest-error constraints by domain and Global versus Pointwise type, alongside rubric score rates across interaction settings.
- Interaction Analysis: Single-turn accuracy exceeds multi-turn accuracy by roughly 10 percentage points on average under strong global constraints.
- Interaction Analysis: Multi-turn refinement remains competitive on local constraints, where feedback can correct infrequent changes, but global consistency erodes on complex tasks.
- GTPO Ablation: TRN stabilizes training, GIN calibrates constraint satisfaction across dialogue history, and TRD yields the best ablation result by using the previous turn’s reward as a baseline.
- Simulator Reliability: 98% simulator reliability was observed across 20 trajectories, while AIS ambiguity and style fidelity averaged 4.7/5 across 10 trajectories.
- Exploration and Reliability: Pass@k increases with more samples, but low pass@1 and substantially worse strict evaluation reveal limited single-try reliability under hard constraints.
7. Conclusion
The paper presents TRIP-Bench as a realistic travel-planning benchmark for long-horizon interaction and introduces GTPO for online multi-turn reinforcement learning. It emphasizes constraint adherence, preference continuity, information integration, goal management, tool orchestration, and diverse user behavior.
- Conclusion: TRIP-Bench evaluates global constraint adherence, long-term planning, multi-tool use, and dynamic user behavior in realistic travel planning.
- Conclusion: The benchmark targets reliable agents that maintain global constraints, coordinate multi-tool execution, and adapt to evolving user preferences over many turns.
- Evaluation Dimensions: Constraint adherence requires compliance with system-level rules, special cases, boundary conditions, and operational limitations.
- Evaluation Dimensions: Preference alignment requires satisfying stated needs while maintaining continuity, reconciling changes or conflicts, and avoiding inconsistent responses.
- Evaluation Dimensions: Information integration combines temporal, spatial, commonsense, environmental, and state-change information into consistent, context-aware solutions.
- Evaluation Dimensions: Goal management progressively turns vague or incomplete requirements into executable objectives while balancing interdependent constraints and trade-offs.
- Evaluation Dimensions: Execution complexity measures multi-step planning, persistent state tracking, conditional branching, tool-call depth, and error-accumulation risk.
- Tool Use: Appropriate tool use preserves functional boundaries, semantic consistency, low-noise outputs, and controlled result sizes.
B.3. Task Difficulty Classification
The benchmark classifies multi-turn task difficulty through nine user behaviors that alter, extend, revise, or clarify goals and plans. These behaviors test whether agents can preserve useful context while adapting to changing interaction demands.
- Behavioral Diversity: Instruction appending adds constraints, preferences, or sub-goals while preserving the original objective.
- Behavioral Diversity: Instruction modification updates part of a prior instruction without rejecting the overall task.
- Behavioral Diversity: Intent redirection changes the fundamental objective and requires replanning while inheriting some earlier instructions.
- Behavioral Diversity: Instruction deletion or rollback cancels a prior requirement or reverts to a historical version.
- Behavioral Diversity: Plan comparison and integration merges unrelated or conflicting goals into one comprehensive plan.
- Behavioral Diversity: Local revision requests targeted edits to a specific portion of model-generated content.
- Behavioral Diversity: Error reporting identifies a model mistake, misunderstanding, or misinterpretation and requests correction.
- Behavioral Diversity: Exploratory inquiry seeks suggestions or possible solutions without requiring the user to adopt them.
B.4. Evaluation Metrics
The evaluation checks whether generated itineraries are structurally valid, complete, feasible in time and space, diverse, and consistent with product requirements.
- Plans must use parseable JSON with correct field names and formatted parameters.
- POIs must exist in the sandbox inventory and belong to the intended planning cities.
- Information completeness requires correct dates and party size, required stays and transport, daily POIs, and hotels except on the return day.
- Temporal reasonableness requires non-overlapping schedules, bounded gaps, plausible durations, opening-hours compliance, intercity buffers, and realistic local travel times.
- Spatial logic, experience diversity, and product consistency assess routing, repetition avoidance, and whether tickets, reservations, quantities, and room capacity meet requirements.
C. Additional Experiments
Additional experiments report DeepSeek-V3.2 performance across single-turn and two multi-turn settings, alongside pass-k evaluation results.
- DeepSeek-V3.2 is evaluated under single-turn, multi-turn with issue reporting, and multi-turn without issue reporting settings.
- Table 5 contains the detailed performance comparison across the three inference settings.
- The paper provides pass-k evaluation results in Figure 5.
D.1. Training Details
The training-details section points readers to the complete GTPO settings and accompanying pass-k and training-curve figures.
- The full detailed GTPO training settings are presented in Table 6.
- Figure 5 presents pass-k performance results.
- Figure 6 presents the training curve.
E. Manual Evaluation Protocol for User Simulator Reliability
The protocol specifies structured itinerary-generation rules and user-simulator behavior, including complete outputs, realistic scheduling, and faithful handling of instruction updates.
- Plans must include transportation, activities, meals, and hotels while avoiding overlaps and excessive gaps.
- Flight and train handling use distinct buffer rules, while local transportation must be scheduled between differing activity locations using route estimates.
- Daily itinerary rules constrain hotel stays, attraction scheduling, restaurant repetition, cuisine diversity, distances, durations, and opening hours.
- Itinerary generation requires a complete JSON trip_plan with fixed field names and structured dates, cities, hotels, activities, IDs, and products.