Source-linked AI summary
TourPlanner: A Competitive Consensus Framework with Constraint-Gated Reinforcement Learning for Travel Planning
Yinuo Wang, Mining Tan, Wenxiang Jiao, Xiaoxi Li, Hao Wang, Xuanyu Zhang, Yuan Lu, Weiming Dong
TL;DR
Travel planning must handle large POI sets, limited reasoning-path exploration, and conflicting hard and soft constraints. TourPlanner addresses these issues with PReSO, CCoT, and sigmoid-gated reinforcement-learning refinement; benchmarks report strong feasibility and preference alignment, while the authors identify challenges in end-to-end RL and reward-model alignment.
Problem
Travel-planning agents must prune large POI sets, explore beyond a single reasoning path, and satisfy hard constraints alongside soft preferences.
Method
TourPlanner combines spatially aware PReSO candidate construction, multi-agent CCoT proposal arbitration, and sigmoid-gated RL that prioritizes soft rewards after hard constraints are met.
Results
TourPlanner achieves a 100% Feasibility Pass Rate across tested LLM backbones, while refined plans reach a Final Pass Rate of 56.1% and Final Surpassing Rate of 30.2%.
Takeaways & Limitations
The framework constructs compact candidate sets, explores feasible plans through consensus, and coordinates feasibility with personalization during refinement.
Takeaways & Limitations
End-to-end RL is difficult for CCoT because effective process rewards are hard to define when travel-planning rewards depend on overall trip quality.
Abstract
from arXiv · showhide
Travel planning is a sophisticated decision-making process that requires synthesizing multifaceted information to construct itineraries. However, existing travel planning approaches face several challenges: (1) Pruning candidate points of interest (POIs) while maintaining a high recall rate; (2) A single reasoning path restricts the exploration capability within the feasible solution space for travel planning; (3) Simultaneously optimizing hard constraints and soft constraints remains a significant difficulty. To address these challenges, we propose TourPlanner, a comprehensive framework featuring multi-path reasoning and constraint-gated reinforcement learning. Specifically, we first introduce a Personalized Recall and Spatial Optimization (PReSO) workflow to construct spatially-aware candidate POIs' set. Subsequently, we propose Competitive consensus Chain-of-Thought (CCoT), a multi-path reasoning paradigm that improves the ability of exploring the feasible solution space. To further refine the plan, we integrate a sigmoid-based gating mechanism into the reinforcement learning stage, which dynamically prioritizes soft-constraint satisfaction only after hard constraints are met. Experimental results on travel planning benchmarks demonstrate that TourPlanner achieves state-of-the-art performance, significantly surpassing existing methods in both feasibility and user-preference alignment.
1 Introduction
TourPlanner addresses three travel-planning challenges—candidate-POI pruning, limited single-path exploration, and joint hard/soft-constraint optimization—through PReSO, CCoT, and constraint-gated reinforcement learning.
- Existing travel-planning agents struggle with oversized POI candidate sets, single reasoning paths, and simultaneous hard- and soft-constraint satisfaction.
- The framework combines candidate construction, multi-path reasoning, and gated refinement to improve itinerary quality, exploration, and feasibility.Its workflow supplies PReSO candidates to CCoT before reinforcement-learning refinement.
- PReSO combines explicit-demand extraction, implicit-preference inference, multi-dimensional recall, and spatial clustering to construct compact, information-rich candidate POI sets.
- CCoT instantiates specialized agents that generate parallel daily proposals and arbitrate them through diversity weighting, peer review, and weighted consensus selection.
- Constraint-Gated RL uses a sigmoid gate to increase soft-constraint reward weight only after hard constraints are satisfied.When hard-constraint satisfaction is below threshold, the gate suppresses soft-constraint rewards; after the threshold, both reward types are optimized.
2 Related Work
Related work establishes benchmarks and increasingly hybrid or agentic approaches for travel planning, while reinforcement learning provides a broader foundation for complex decision-making and reasoning optimization.
- TravelPlanner introduced a large-scale sandbox benchmark with travel data and tool access, exposing weak grounding and limited constraint handling in existing LLMs.
- Hybrid methods combine formal constraint modules with LLM reasoning to improve the reliability and constraint satisfaction of generated plans.Hao et al. integrate satisfiability solvers, while TRIP-PAL combines automated planners with LLM reasoning.
- Agentic systems extend travel planning through modular tool use, memory, wide-horizon reasoning, simulated evaluation, and multi-agent management of implicit intent.
- Reinforcement learning methods have progressed from preference alignment through RLHF to approaches targeting training stability and large-scale reasoning optimization.
- These reinforcement-learning advances position RL as a tool for complex decision-making tasks, including travel planning.
3 TourPlanner
TourPlanner addresses candidate-POI scale, limited single-path exploration, and joint hard- and soft-constraint optimization through PReSO, CCoT, and constraint-gated reinforcement learning. Its pipeline combines spatially aware retrieval, competing specialized proposals, consensus arbitration, and reward gating based on hard-constraint satisfaction.
- Personalized Recall and Spatial Optimization (PReSO): PReSO constructs a spatially compact candidate-POI set by combining explicit requirements, inferred preferences, multidimensional recall, and spatial clustering.Cluster centroids anchor nearby accommodations and restaurants, while cluster labels enrich the planning input.
- Competitive consensus Chain-of-Thought (CCoT): CCoT replaces single-path reasoning with specialized agents that generate parallel daily proposals optimized for distinct objectives and ranked priorities.The agent team explicitly represents diverse preferences such as culture, gourmet experiences, and budget.
- Competitive consensus Chain-of-Thought (CCoT): The proposal-generation process uses a shared base routing skeleton, independent agent refinement, removal of previously selected items, and rule validation.This parallel exploration produces competing daily plans before arbitration.
- Competitive consensus Chain-of-Thought (CCoT): CCoT arbitrates competing proposals through diversity weighting, parallel peer review, and weighted consensus selection.Diversity weights reward unique proposals, while peer evaluations capture objective-specific quality, priorities, and feasibility.
- Constraint-Gated Reinforcement Learning: Constraint-gated reward optimization increases the soft-constraint reward only after hard-constraint satisfaction reaches the threshold.When the hard-constraint score η is below τ, α approaches 0 and the objective focuses on Rhard; once η reaches τ, α increases toward 1 and incorporates Rsoft.
4 Experiments
Experiments evaluate TourPlanner on the TripTailor benchmark, including overall performance, route efficiency, feasibility, recall, and component ablations across planning methods and LLM backbones.
- Main Results: TourPlanner exceeds 88% Macro Pass Rate across all tested LLM backbones, whereas baseline methods often remain below 30%.Macro Pass Rate measures simultaneous satisfaction of complex travel constraints.
- Main Results: TourPlanner reduces Average Route Distance Ratio from as high as 5.98 for Direct Planning with GPT-4o to 2.15.The framework produces more spatially efficient and coherent itineraries.
- Main Results: Across all tested LLM backbones, TourPlanner achieves a 100% Feasibility Pass Rate and a Final Surpassing Rate above 20%.After RL refinement, the method reaches a 56.1% Final Pass Rate and 30.2% Final Surpassing Rate.
- Analysis: With GPT-4o, PReSO achieves 42.26% recall versus TripTailor’s 27.83%, a +14.43% improvement in identifying ground-truth travel elements.The comparison covers hotels, restaurants, and POIs across three backbone models.
- Analysis: Removing CCoT lowers Macro Rationality to 84.9% and Final Pass Rate to 47.8%, while 4-6 agents outperform 3 and 10-agent scaling shows diminishing returns.With 10 agents, Macro Rationality plateaus at 90.1% and Final Pass Rate slightly regresses.
- Analysis: Direct refinement and vanilla RL perform worse than constraint-gated RL, demonstrating the importance of gating hard- and soft-constraint rewards.The gating mechanism supports simultaneous optimization of common-sense constraints and personalized preferences.
5 Conclusion
TourPlanner combines spatially aware POI pruning, multi-path consensus reasoning, and constraint-gated reinforcement learning to improve itinerary quality. Experiments report state-of-the-art performance in both feasibility and alignment with user preferences.
- PReSO uses three-branch recall and spatial clustering to construct a compact, information-rich candidate POI set.
- CCoT uses multi-agent arbitration to resolve conflicts and explore the feasible solution space.
- A sigmoid-based gating mechanism prioritizes soft-constraint rewards only after hard constraints are sufficiently satisfied.
- TourPlanner achieves state-of-the-art performance, outperforming existing methods in feasibility and alignment with user preferences.
Limitations
The authors identify challenges in applying end-to-end reinforcement learning to CCoT and limitations in the reward models explored.
- CCoT's complex day-by-day iterative generation process makes end-to-end reinforcement-learning optimization challenging.
- Defining an effective process reward function is difficult because travel-planning rewards typically depend on the overall quality of the entire trip.
- The study's reward-model exploration is limited and primarily adopts methodologies from previous works.
- Future research should align reward mechanisms more closely with user preferences to achieve a higher surpassing rate.
LLM statement
The paper uses LLMs for language refinement and discusses sequence-level policy optimization and notation definitions. The supplied passages also indicate that Table 3 covers the paper's principal notations.
- LLMs were used to polish grammar, improve clarity, and enhance the academic style of the writing.
- GSPO introduces a sequence-level importance ratio rather than a token-level ratio to exclude excessively off-policy samples from gradient estimation.
- Table 3 presents the principal notations and definitions adopted in the paper.
C Additional Experiment Details
The paper summarizes its method hyperparameters in Table 4.
- Table 4 summarizes the hyperparameters involved in the method.
C.2 Details of Metrics
The section defines metrics for personalization, feasibility, rationality, route efficiency, and overall plan quality. These measures combine validity checks, itinerary criteria, route compactness, and comparison with human or reference plans.
- Feasibility Pass Rate marks plans infeasible when they contain hallucinations or cannot match POIs within the sandbox environment.
- Rationality Pass Rate requires satisfying all five criteria covering restaurant and attraction diversity, visit duration and time, and budget limits.
- Average Route Distance Ratio compares generated and reference-plan distances, with lower values indicating more efficient and compact routes.A plan’s average distance is obtained by averaging daily consecutive-POI segment distances across the itinerary.
- Final Pass Rate requires both Feasibility and Rationality passes while limiting total route length to 1.5 times the reference plan.
- Final Surpassing Rate measures whether a generated plan matches or exceeds the personalization quality of a human-created plan.It uses Gemini-3-Pro as an LLM-based judge comparing the generated and real plans.
C.4 Baselines
The section describes the evaluated model backbones and planning baselines, alongside the hard- and soft-constraint reward components used to assess plans. These components separately represent viability and preference or efficiency quality.
- LLM Backbones: The backbone comparison includes GPT-4o and three open-source models: Qwen3-235B-A22B-Instruct, Qwen3-30B-A3B-Thinking, and DeepSeek-R1.Qwen3-30B-A3B-Thinking and DeepSeek-R1 are designated thinking models.
- Planning Approaches: The planning comparison covers Direct Planning, ReAct Planning, and the structured TripTailor Workflow.TripTailor decomposes planning into transportation selection, attraction ranking, itinerary generation, nearby dining and accommodation integration, and final output.
- Hard-Constraint Reward: The hard-constraint reward averages Feasibility and Rationality scores to evaluate fundamental plan viability.Feasibility covers sandbox validity and information completeness, while Rationality covers itinerary coherence criteria.
- Hard-Constraint Reward: Feasibility checks entity validity and completeness of essential prices, schedules, and transportation details.
- Soft-Constraint Reward: The soft-constraint reward sums normalized budget, route-efficiency, and preference-alignment scores.Route efficiency compares generated and reference distances, while preference alignment uses a reward model for implicit user preferences.
C.5.3 Training Infrastructure
The section specifies the infrastructure and prompting procedures used to train and operate TourPlanner. It covers model fine-tuning, reinforcement learning, budget heuristics, agent-role generation, daily planning, and plan evaluation.
- Training Infrastructure: Supervised fine-tuning uses Qwen2.5-3B-Instruct on 8 NVIDIA RTX 4090 GPUs with a 4096-token maximum sequence length.Training uses learning rate 1e-5, weight decay 0.01, 2 epochs, batch size 4, and 2 gradient accumulation steps.
- Training Infrastructure: Reinforcement-learning fine-tuning uses 32 NVIDIA H800 GPUs, 8 responses per prompt, and 3 training epochs.The setup uses dynamic batching and sequence parallelism, with prompt and response limits of 30,000 and 8,000 tokens.
- Prompting Pipeline: The prompting pipeline extracts structured travel details such as dates, times, duration, cities, requirements, budget, and restaurant type.Fields are enclosed in square brackets, with unspecified details inferred except for Other Requirements and Restaurant Type, which may remain blank.
- Agent Construction: The system generates 4–6 specialized agents with measurable objectives, ranked priorities, and distinct personalities based on query complexity.Example roles include cultural_scholar, foodie_explorer, and budget_manager; objectives may target experience scores, meal prices, or daily costs.
- Daily Planning: Per-agent daily plans require source integrity, cluster-based selections, valid visit durations and hours, sequential activities, and at least 30 minutes of transfer time.Arrival and departure days have special transportation and hotel check-in or check-out rules, while ordinary days target 7–8 active hours.
- Plan Evaluation: Final evaluation considers experiences, itinerary intensity, cuisine, accommodations, transportation, and total budget.
E Case Study
The case study presents an example user query together with travel-planning results generated by TourPlanner. Table 5 is organized around the query and its corresponding planning output.
- Table 5 presents an example of a user query and the corresponding travel-planning results generated by TourPlanner.