Source-linked AI summary
An Empirical Evaluation of Cross-City POI Recommendation on a Large-Scale Benchmark
Peibo Li, Yang Song, Hao Xue, Maarten de Rijke, Flora D. Salim
TL;DR
Cross-city POI recommendation must transfer preferences to unfamiliar destinations with unseen POI inventories, but prior evidence is limited by small and semantically sparse benchmarks. The paper evaluates representative methods on Trip World, a worldwide benchmark with enriched metadata and reviews, and finds that destination-region priors, weak scalability, and ineffective semantic integration constrain performance. A diagnostic agentic pilot further shows that naive next-POI adaptation trails popularity despite relevant semantic signal in the data.
Problem
Cross-city recommendation requires transferring preferences from a home city to an unfamiliar destination with a largely unseen POI inventory, while prior benchmarks provide limited large-scale, realistic, and semantically rich evidence.
Method
The paper evaluates popularity-based, sequential, neural, semantic-aware, and agentic methods on Trip World, using worldwide out-of-town check-ins, enriched POI metadata, and user reviews.
Results
Hometown-aware methods rely more on destination-region priors than transferable user preferences, the simplest model is among the strongest at this scale, and semantic integration yields little benefit.
Takeaways & Limitations
Cross-city POI recommendation needs task-specific designs for preference transfer, semantic grounding, and scalable reasoning over unseen destination inventories.
Takeaways & Limitations
Popularity dominates overall partly because candidate matching is approximate and most queries have popular ground truth, while semantic reasoning shows headroom on long-tail queries.
Abstract
from arXiv · showhide
Cross-city point-of-interest (POI) recommendation is crucial for navigating unfamiliar urban environments, yet its progress has historically been constrained by data limitations. Using the recently proposed large-scale benchmark Trip World, we empirically re-examine whether conclusions drawn on small prior benchmarks still hold under worldwide coverage, low home-destination region overlap, and large, semantically rich POI inventories. Our evaluation surfaces three bottlenecks of representative state-of-the-art methods: (1) hometown-aware models appear to rely more on destination-region priors than on user-specific preference transfer; (2) their accuracy-efficiency trade-off degrades at this scale, where the simplest model is among the strongest; and (3) existing mechanisms for integrating semantic metadata yield little benefit. We further include a diagnostic pilot on agentic methods adapted from next-POI recommendation, finding that naive adaptation trails a simple popularity prior even though the relevant semantic signal is present in the data. These results highlight the need for task-specific designs that support cross-city preference transfer, semantic grounding, and scalable reasoning over unseen destination inventories.
1 Introduction
Cross-city POI recommendation addresses travel in unfamiliar cities, where destination POIs are unseen and routine-based histories may not transfer. The paper uses Trip World to evaluate whether established methods generalize under worldwide coverage and richer semantic information.
- Travelers face unseen destination POIs, sparse trajectories, shifting tourism preferences, and home histories that may not transfer.Cross-city recommendation must transfer preferences from a home city to a largely unseen destination inventory.
- Prior benchmarks are often single-city, geographically narrow, small, or poor in semantic metadata, limiting realistic cross-city evaluation.Some datasets use coordinates and coarse categories, while review-based interactions may not represent real visit sequences.
- Trip World supplies worldwide out-of-town check-ins, enriched POI metadata, and user reviews for unified evaluation of classical, neural, semantic-aware, and agentic methods.Its scale supports testing mobility history, destination context, POI semantics, and travel-specific behavior beyond proximity or popularity.
- The evaluation identifies reliance on destination-region priors, degraded accuracy-efficiency trade-offs, and limited gains from semantic-integration mechanisms.The simplest model is among the strongest, while the most compositional model ranks lowest at much higher cost.
2 Related Work
Related work spans intra-city sequence recommendation and cross-city preference transfer, but most prior methods assume richer localized history than unfamiliar-city travel provides. Existing comparison datasets are also small, geographically limited, or based on proxy interactions.
- Intra-city methods predict intermediate POI sequences within one region using localized histories, but degrade under out-of-town cold start.Representative approaches include dual-LSTM, graph, Transformer, and neural point-process models.
- Cross-city recommenders transfer preferences across disjoint POI spaces using latent spaces, graphs, causal approaches, or meta-learning.SPOT-Trip combines trip modeling and cross-city transfer through a POI attribute knowledge graph and neural ODEs for preference drift.
- Only Foursquare and Yelp among the summarized prior datasets contain genuine cross-city activity, while both remain small and largely US-centric.Foursquare lacks auxiliary POI information, whereas Yelp check-ins are review artifacts and include substantial phantom travel.
3.1 Experimental setup
The experimental setup defines out-of-town trip prediction over intermediate POIs, compares seven baseline families, and evaluates predictions with overlap and order-sensitive metrics. Trip World is processed into user-level train, validation, and test partitions.
- Task: Given hometown history and trip context, models predict the ordered intermediate POIs between an origin and destination.Inputs include hometown history, origin, destination, target region, and trip length; semantic variants additionally use categories, coordinates, and review signals.
- Dataset comparison: Table 1 summarizes prior-dataset coverage, check-ins, and whether records are genuine, reconstructed, or proxy-based.The caption states that four city-specific datasets cover one region and contain only hometown check-ins by construction.
- Trip World statistics: Table 2 reports processed Trip World statistics, including check-ins and the mean number of POIs per out-of-town trip.Avg |c_o| denotes the mean number of POIs per out-of-town trip.
- Baselines: Seven baselines include four methods without hometown information and three hometown-aware methods, with KDDC and PPROC using the SPOT-Trip backbone.The Popularity baseline ranks POIs by global training-set visit frequency.
- Metrics: F1 measures predicted-ground-truth POI overlap, while Pairs-F1 additionally rewards correct visiting order.Both metrics are computed only over intermediate POIs, excluding the fixed origin and destination.
- Data processing: The dataset excludes users with fewer than three out-of-town records and trips shorter than 1 hour or longer than 30 days.Remaining users are partitioned into training, validation, and testing sets at 80%/10%/10%.
3.2 Results and Discussion
On Trip World, hometown-aware methods do not outperform out-of-town-only methods, while the simplest model is among the strongest and the costliest compositional model ranks lowest. A tested review-rating relation also provides little benefit.
- Transferable preferences, or region priors?: Hometown-aware methods do not outperform methods using only out-of-town signals on Trip World, unlike on Foursquare and Yelp.
- Transferable preferences, or region priors?: 30.9% of destination regions also appear as home regions in Trip World, indicating a low-overlap evaluation regime.
- Transferable preferences, or region priors?: Shuffling hometown histories leaves SPOT-Trip F1 unchanged, with relative change below 0.55% across three seeds.
- Accuracy–efficiency trade-off at scale.: MatTrip attains the best accuracy, whereas SPOT-Trip ranks lowest and is the costliest model to train on Trip World.Trip World contains 336,102 POIs across 890 regions; the authors cannot fully separate architecture from tuning or implementation.
- Semantic integration: SPOT-Trip no-review outperforms SPOT-Trip full, indicating that the single review-derived rating relation does not help in this test.The paper stresses that this probes one semantic-integration mechanism rather than semantic metadata in general.
3.3 Can Agentic Methods Transfer to Cross-City Recommendation?
The pilot adapts LLM agents to in-candidate cross-city POI selection and finds that naive transfer trails popularity, while semantic signals fail to improve selections despite being present in the data.
- Adaptation and setup: LLMMove directly ranks candidates, whereas AgentMove adds memory and world-model reasoning using Claude Haiku 4.5.
- Adaptation and setup: Open-vocabulary generation over 336,102 POIs is replaced with selecting and ordering 100 candidates containing the ground-truth intermediates.
- Semantic signals: Review summaries describe candidate POIs, while user-taste profiles summarize preferred cuisines, venue types, vibe, and price level from hometown visits.
- Results: 0.0718 vs. 0.1069 F1: every agentic variant trails in-candidate Popularity, and AgentMove underperforms the lighter LLMMove.Adding candidate popularity to the prompt helps more than semantic signals but still does not close the gap.
- Results: On destinations with ≤30 training trips, removing either semantic signal improves F1, with no semantics strongest among LLM configurations.The ground-truth intermediates match hometown POI categories 40.6% of the time versus 25.3% for popularity-matched negatives.
- Implications: Long-tail queries reveal headroom: agents exceed popularity when the ground truth is unpopular, motivating prior-anchored semantic refinement and scalable retrieval.The paper also identifies vocabulary alignment and grounding over unseen destination inventories as directions for cross-city-native methods.
4 Conclusion
Using Trip World, the paper re-examines cross-city POI recommendation at worldwide scale and identifies bottlenecks in preference transfer, efficiency, and semantic integration. Its diagnostic agentic pilot also trails popularity, motivating task-specific designs for transfer, grounding, and scalable reasoning.
- 4 Conclusion: Under low home–destination region overlap, hometown-aware methods rely on per-region popularity priors rather than transferable user preferences.Replacing a user’s hometown history with another user’s leaves SPOT-Trip essentially unchanged.
- 4 Conclusion: At this scale, MatTrip is among the strongest while SPOT-Trip ranks lowest at the highest cost.
- 4 Conclusion: A common semantic-metadata integration mechanism yields no benefit.
- 4 Conclusion: The adapted agentic methods trail a simple in-candidate popularity prior even though relevant semantic signal is present.
- 4 Conclusion: Progress requires task-specific designs for cross-city preference transfer, semantic grounding, and scalable reasoning over unseen destination inventories.