Source-linked AI summary
Fine-tuning LLMs for Tourist Trajectory Prediction using Field Experiment Data
Tatsuya Amano, Hirozumi Yamaguchi
TL;DR
Tourist mobility interventions require predictions that account for context and generalize beyond observed scenarios, while traditional models have limited contextual and data efficiency. This paper fine-tunes Llama-3.1-8B on 566 local trajectories represented as conditional, structured text, achieving 49.1% next-POI accuracy and strong performance in undersampled contexts. The resulting predictive capability provides a foundation for future counterfactual mobility-intervention analysis, but does not itself establish causal effects.
Problem
Existing tourist behavior models struggle to incorporate context and generalize to unobserved scenarios, while real-world validation of fine-tuned LLMs remains limited.
Method
The paper fine-tunes Llama-3.1-8B on 566 Wakayama Castle Park trajectories, conditioning structured visit sequences on persona, environmental conditions, and prior history.
Results
49.1% next-POI accuracy was achieved on real-world Wakayama Castle Park trajectories, with strong generalization to undersampled contexts such as rainy days.
Takeaways & Limitations
The results establish the feasibility of fine-tuned LLMs as predictive behavior models and provide a foundation for future counterfactual mobility-intervention simulations.
Takeaways & Limitations
Predictive accuracy does not validate causal claims about behavioral changes under hypothetical mobility interventions.
Abstract
from arXiv · showhide
Evaluating mobility interventions at tourist destinations requires predicting visitor behavior under varying conditions. Traditional methods struggle because tourist decisions depend heavily on context like weather and fatigue, yet models cannot generalize to unobserved scenarios. Large Language Models offer a solution by encoding commonsense knowledge about human behavior from pretraining, enabling reasoning about context-dependent decisions, while natural language representation flexibly integrates heterogeneous information. Fine-tuning on local trajectories adapts this general understanding to destination-specific patterns. We validate this approach using 566 trajectories from Wakayama Castle Park, Japan. Our fine-tuned Llama-3.1-8B achieves 49.1% next POI accuracy and maintains strong performance on undersampled scenarios like rainy days, demonstrating effective generalization. This establishes LLMs as high-fidelity behavior models for context-dependent tourist prediction, providing groundwork for counterfactual analysis of mobility interventions.
Introduction
Tourist destinations struggle to redistribute visitors because mobility decisions depend on physical layout, context, and accumulated experience. The paper proposes fine-tuning LLMs on local trajectories to combine general behavioral knowledge with destination-specific patterns and predict undersampled scenarios.
- Motivation: Uneven visitor distribution concentrates crowds at popular landmarks while nearby cultural sites remain underused.Physical barriers, attraction layout, and limited awareness of alternatives contribute to concentrated flows.
- Mobility interventions: Mobility services such as shuttles, e-bikes, and walking support devices are being deployed to connect distributed attractions.Field experiments at Wakayama Castle Park included three mobility services linking southern and northern facilities.
- Research challenge: Predicting whether mobility interventions will change routes, exploration, and dwell times remains difficult for destination planners.Reliable forecasts are needed to assess whether visitors redistribute their activity or continue visiting the same popular locations.
- Research challenge: Traditional models struggle with memory, contextual information, scarce destination data, and unobserved scenarios.They cannot readily use commonsense knowledge to infer responses to changed conditions or new mobility options.
- Approach: Fine-tuning LLMs combines pretrained behavioral knowledge with destination-specific trajectory learning and text-based contextual inputs.The approach represents weather, fatigue, mobility availability, and preferences without manual feature engineering, supporting interpolation in undersampled scenarios.
- Evaluation: 49.1% next-POI accuracy was achieved by fine-tuned Llama-3.1-8B on 566 Wakayama Castle Park trajectories.The model also maintained strong performance on rare contexts such as rainy days and generated coherent multi-step trajectories.
Related Work
Tourist next-POI prediction has progressed from statistical models to deep learning, but existing approaches require substantial data and generalize poorly to unobserved contexts. LLM-based behavioral modeling offers commonsense reasoning, yet validation on fine-grained real-world tourist trajectories remains limited.
- Tourist mobility prediction: Next-POI research primarily uses location-based social-network check-ins and has evolved from Markov models to deep learning.The literature frames tourist mobility prediction as a progression toward more complex predictive architectures.
- Classical approaches: Markov models impose memoryless transitions and become data-sparse at higher orders, limiting their ability to capture long-term preferences.Discrete choice models represent rational decisions explicitly but face the large state space of possible POIs.
- Deep learning limitations: Deep learning models achieve higher accuracy but require massive datasets, cannot process contextual text well, and generalize poorly to unobserved scenarios.These requirements are especially problematic for individual destinations.
- LLMs for behavior modeling: LLM agents can generate believable routines and interactions using memory, reflection, planning, and commonsense reasoning.Prior work demonstrated behavioral modeling beyond traditional natural-language-processing tasks.
- LLMs for mobility: Textual descriptions have enabled mobility prediction under public-event conditions.MPE provides an example of using event information in textual form.
- Research gap: Validation of fine-tuned LLMs on fine-grained field-experiment data from specific destinations remains limited.It is unresolved whether LLMs can model context-dependent tourist decisions and generalize to unobserved scenarios within a location.
Methodology
The methodology formulates tourist movement as conditional sequence generation, conditioning each predicted visit on tourist attributes, environmental conditions, and prior history. It builds a unified, context-rich trajectory dataset and represents visits as structured text for LLM fine-tuning and next-POI generation.
- Problem formulation: Tourist trajectory prediction is formulated as conditional sequence generation over time-ordered POI visits.Each visit records a POI, time, area, and category.
- Problem formulation: The model conditions predictions on tourist persona, environmental conditions, and visit history before the current step.Persona includes age, gender, and group type; environmental conditions include weather and time.
- Data collection and preprocessing: The dataset contains 566 trajectories collected at Wakayama Castle Park across GPS-tracked and QR-code participants.The park includes 68 POIs, while 31 additional POIs were added from OpenStreetMap for broader coverage.
- Data collection and preprocessing: GPS traces were stop-detected and matched to POIs, while QR sequences were deduplicated and both sources were unified with weather and visit attributes.Exit-survey demographics were collected, and 18% of missing attributes were inferred using GPT-4o.
- Model architecture and representation: Llama-3.1-8B was fine-tuned using special tokens for POIs, areas, categories, and structural trajectory tags.Structured text preserves timestamp, action, area, category, and POI information for sequential modeling.
- Training and inference: At inference, the model receives visit history through the last area tag and generates the subsequent category and POI tags.Training generates complete trajectories from persona and environment using supervised fine-tuning.
Experiments
Experiments evaluate fine-tuned Llama-3.1 against statistical, language-model, and sequence-level baselines on tourist trajectories from Wakayama Castle Park. The model achieves stronger next-POI performance, adapts to undersampled contexts, and generates coherent trajectories.
- Experimental Setup: 566 tourist trajectories were evaluated using tourist-level training and test splits, with comparisons against Markov, Hidden Markov, GPT-4o, and Llama-3-Swallow baselines.The dataset used 453 tourists for training and 113 for testing; evaluation included Accuracy@1 and sequence-level coherence metrics.
- Prediction Performance: 15.3% accuracy was achieved by fifth-order Markov models, compared with 9.0% for first-order Markov and 11.0% for Hidden Markov models.The fifth-order model suffered data sparsity with 68 POIs.
- Prediction Performance: 49.1% POI accuracy and 55.4% category accuracy were achieved by fine-tuned Llama-3.1, outperforming all baselines.Llama-3-Swallow achieved 39.8% POI and 45.8% category accuracy.
- Sequence Generation Quality: 31.2% 4-gram overlap, 25.8% BLEU, and 47.0% normalized Levenshtein distance characterized sequence generation, with anomalous outputs in 5.4% of generations.Generated trajectories averaged 7.2 POIs, close to the real-data average of 7.7.
- Contextual Generalization: 41.7% accuracy on 12 rainy test samples exceeded the 8.3% achieved by Markov models, while lunch-time dining predictions reached 62.3% category accuracy.The model increased indoor predictions and reduced garden visits during rainy conditions, while adapting predictions temporally around lunch.
Conclusion
Fine-tuned LLMs show feasibility for tourist trajectory prediction, while important limits remain for causal evaluation, inferred attributes, novel POIs, and major infrastructure changes.
- Conclusion: 49.1% next-POI accuracy was achieved on 566 real-world Wakayama Castle Park trajectories, with strong generalization to undersampled contexts such as rainy days.The model also generated coherent multi-step trajectories reflecting realistic visitor behavior.
- Conclusion: Predictive capability provides a foundation for counterfactual generation but does not validate causal claims about mobility interventions.Causal evaluation requires future validation against A/B tests or randomized field trials.
- Conclusion: 18% of missing persona attributes were inferred using GPT-4o, although manually verified subsets showed minimal impact.
- Conclusion: The model cannot predict behavior at entirely new POIs without descriptions or reason about major infrastructure changes.The paper identifies retrieval-augmented generation and few-shot adaptation as possible extensions.