PaperlayerView exact revision on arXiv

Source-linked AI summary

Where Would I Go Next? Large Language Models as Human Mobility Predictors

Xinglei Wang, Meng Fang, Zichao Zeng, Tao Cheng

arXiv:2308.15197v2cs.AIcs.SIphysics.soc-ph

TL;DR

Human mobility is difficult to predict accurately and interpretably, and whether LLMs can model mobility data remains unknown. LLM-Mob structures mobility data for long- and short-term dependencies, time-aware prediction, and context-inclusive prompting; on two public datasets, it achieves state-of-the-art predictive performance and superior interpretability.

  • Problem

    Mobility prediction remains difficult because of stochasticity and complex spatiotemporal dependencies, while deep models still have accuracy and interpretability limits and LLMs’ ability to model mobility is unknown.

    Source: §1 · p. 1

  • Method

    LLM-Mob formats mobility data to capture long- and short-term dependencies and support time-aware prediction, then uses context-inclusive prompts for LLM reasoning.

    Source: §7 · p. 6

  • Results

    On two real-world publicly available mobility datasets, LLM-Mob demonstrates state-of-the-art predictive performance and superior interpretability.

    Source: §7 · p. 7

  • Takeaways & Limitations

    The experiments demonstrate the promising utility of LLMs for human mobility prediction.

    Source: §1 · p. 2

    The current method calls the OpenAI API separately for each test sample, making it inefficient and hard to apply to large-scale predictions.

    Source: §6.2 · p. 6

Abstract

from arXiv · show

Accurate human mobility prediction underpins many important applications across a variety of domains, including epidemic modelling, transport planning, and emergency responses. Due to the sparsity of mobility data and the stochastic nature of people's daily activities, achieving precise predictions of people's locations remains a challenge. While recently developed large language models (LLMs) have demonstrated superior performance across numerous language-related tasks, their applicability to human mobility studies remains unexplored. Addressing this gap, this article delves into the potential of LLMs for human mobility prediction tasks. We introduce a novel method, LLM-Mob, which leverages the language understanding and reasoning capabilities of LLMs for analysing human mobility data. We present concepts of historical stays and context stays to capture both long-term and short-term dependencies in human movement and enable time-aware prediction by using time information of the prediction target. Additionally, we design context-inclusive prompts that enable LLMs to generate more accurate predictions. Comprehensive evaluations of our method reveal that LLM-Mob excels in providing accurate and interpretable predictions, highlighting the untapped potential of LLMs in advancing human mobility prediction techniques. We posit that our research marks a significant paradigm shift in human mobility modelling, transitioning from building complex domain-specific models to harnessing general-purpose LLMs that yield accurate predictions through language instructions. The code for this work is available at https://github.com/xlwang233/LLM-Mob.

1 Introduction

LLM-Mob explores whether large language models can predict human mobility, organizing movement data and prompts to address the task. On two public datasets, it reports superior predictive performance and interpretability.

1 Introduction

  • Human mobility prediction supports downstream applications and studies of societal challenges, including recommendation, traffic optimization, urbanization, segregation, and epidemics.The introduction presents LLM-Mob as an initial step toward general-purpose LLM-based mobility modelling and a way to repurpose LLMs through data organization and prompting.Source: §1 · pp. 1–2
  • Human mobility prediction remains difficult because movement is regular yet stochastic, and existing deep models have limited accuracy and explanations.The authors connect these shortcomings to the interpretability and applicability of such models in real-world settings.Source: §1 · p. 1
  • Whether LLMs can model human mobility data remains unknown, despite their strong performance in language tasks.The paper frames this as an open question about extending LLMs to mobility prediction.Source: §1 · p. 1
  • LLM-Mob organizes movement data into historical and context stays, uses target-time information, and applies context-inclusive prompts.These choices address long- and short-term dependencies and aim to support time-aware prediction and interpretable outputs.Source: §1 · pp. 1–2
  • LLM-Mob achieves superior predictive performance and interpretability in experiments on two public human mobility datasets.The experiments compare the method with state-of-the-art models and analyze prediction results.Source: §1 · p. 2

2 Related work

Prior mobility prediction research progressed from Markov models to deep sequence models, while LLM use for mobility remained unexplored. This paper takes an initial step by applying in-context learning to the task.

2.1 Human Mobility Prediction

  • Deep learning approaches address long-range dependencies, but performance gains have been incremental and predictions remain difficult to fully explain.Earlier Markov models assumed location visits were Markovian; later RNNs and Transformers model more complex sequential dependencies.Source: §2.1 · p. 2

2.2 Large Language Models

  • LLMs are large pretrained Transformer language models, and in-context learning uses instructions or demonstrations to elicit outputs without additional training.Prompt design, including chain-of-thought strategies, is presented as important to in-context learning.Source: §2.2 · p. 2, In-context learning · p. 2

In-context learning.

  • The paper makes an initial attempt to use LLMs for human mobility modelling and prediction through an in-context learning process.The authors state that prior research had not explored LLM use in human mobility studies.Source: In-context learning · p. 2

3 Preliminaries

The preliminaries represent mobility trajectories as sequences of stays and define next-location prediction as forecasting the place at the next time step from recent stays.

3 Preliminaries

  • The preliminaries introduce the notations and formulate the next-location prediction problem.The section sets up the terms used to describe mobility data and the prediction task.Source: §3 · p. 2

Terms and Notations.

  • A stay is represented by its start time, day of week, duration, and unique place identifier.A trajectory is preprocessed into a sequence of stays in which a person remains stationary for a period.Source: Terms and Notations · pp. 2–3

Next Location Prediction.

  • Given the user's sequence of Q stays through time n, the task is to predict the place identifier at the next time step.The sequence length Q determines how much past information the predictive model considers.Source: Next Location Prediction · p. 3

4 Methodology

LLM-Mob formats mobility histories into long-term and short-term stay context, incorporates the target time, and uses context-inclusive prompts to elicit ranked next-place predictions with explanations.

4 Methodology

  • The workflow formats mobility data into historical and context stays, uses the target stay’s time, and feeds the resulting prompt to an LLM for predictions and reasons.Historical stays represent longer-term dependencies, while context stays capture recent activity.Source: §4 · p. 3

Capturing Long-term and Short-term Dependencies.

Target Time Information for Time-Aware Prediction.

  • Time-aware prediction uses the target stay’s time to predict the place at that specific future time, rather than predicting a next place without temporal information.The paper motivates this as a more realistic framing of next-location prediction.Source: Target Time Information for Time-Aware… · p. 3

4.2 Context-Inclusive Prompting

  • Context-inclusive prompts combine instructions and mobility data, building on Chain-of-Thought and Plan-and-Solve strategies to help LLMs interpret the task.The prompt template’s annotated subparts explain their functions; those annotations are not included in the prompt.Source: §4.2 · p. 3, Figure 4 · p. 4

Describe the data.

  • The prompt describes the three formatted data types so the LLM can process them in the context of human mobility prediction.Source: Describe the data · p. 4

Specify the Number of Output Places.

  • The model outputs k probable places in descending probability order, allowing evaluation of accuracy and cumulative gain through rank k.The paper reports that output length can influence performance.Source: Specify the Number of Output Places · p. 4

Guide the Model to “think”.

  • The prompt explicitly directs the model to consider historical patterns, recent patterns, and target time, using instructions and context rather than demonstration examples to guide reasoning.Source: Guide the Model to “think” · p. 4

Ask for Explanation.

  • The prompt asks for both a prediction and its supporting reason to make results interpretable and further encourage reasoning.The paper relates this explanation request to the reasoning-generation step in Plan-and-Solve prompting.Source: Ask for Explanation · p. 4

5 Experiments

The experiments evaluate LLM-Mob on two differently sourced mobility datasets, using GPT-3.5 and comparisons with classical and deep-learning baselines.

5.1 Datasets and Preprocessing

  • Experiments use the Geolife GNSS tracking dataset and Foursquare New York City check-ins, following prior preprocessing steps for filtering users, forming stays, and splitting data.Dataset statistics after preprocessing are reported as per-user means and standard deviations.Source: §5.1 · p. 4

Implementation Details.

  • The experiments use GPT-3.5 with temperature 0 and set historical and context stay counts to M = 40 and N = 5.The authors describe these counts as empirical values that can be adapted to specific needs.Source: Implementation Details · p. 4

Baselines.

  • Comparisons include a first-order Markov chain, LSTM-based models, DeepMove, MobTcast, and MHSA.Source: Baselines · pp. 4–5

Evaluation Metrics.

  • Evaluation covers top-k accuracy, visit-weighted F1, and nDCG@10, with nDCG based on relevance at each ranked position.In this location-prediction setting, relevance is binary and marks whether a ranked item is the ground-truth next location.Source: Evaluation Metrics · p. 5

5.3 Accuracy Comparison

LLM-Mob outperforms all baseline models across the reported metrics; target-time information improves performance at k = 10, and k = 1 further improves Acc@1 and F1.

5.3 Accuracy Comparison

  • LLM-Mob significantly outperforms all baseline models on every reported metric.With k = 10, the target-time variant outperforms the variant without target-time information; k = 1 further improves Acc@1 and F1 by a large margin.Source: §5.3 · p. 5

5.4 Ablation Study

On Geolife, LLM-Mob outperformed all five ablation variants, with removing historical stays causing the largest performance drop. The study tested data-formatting components and prompting strategies under k = 1 time-aware prediction.

5.4 Ablation Study

  • NoHistory, NoContext, and NoTime removed historical stays, context stays, and target-time information, respectively.NoGuide removed the instruction to guide the model's thinking, while NoReason removed the requirement to provide supporting reasons.Source: §5.4 · p. 5
  • The five variants were evaluated on Geolife under the k = 1 time-aware prediction setting.The study assessed the effects of data-formatting components and prompting strategies.Source: §5.4 · p. 5
  • The full model consistently outperformed all five variants, and NoHistory caused the largest performance drop.The ablation was conducted on Geolife under k = 1 time-aware prediction.Source: Figure 5 · p. 6, §5.4 · p. 6

5.5 Case Study

The case study reports an example where LLM-Mob predicted the next place and provided reasons, while MHSA did not explain its result. Robustness tests found no significant performance variation across tested temperatures, prompts, and LLMs.

5.5 Case Study

  • In case 1, LLM-Mob predicted place 1 at k = 1 and included it in its top 10 at k = 10, while MHSA missed it.LLM-Mob provided detailed, rational reasons, whereas MHSA could not explain its result.Source: §5.5 · p. 6
  • Case 2 showed duplicated top-10 place IDs, attributed mainly to user 13 having few unique visited places, and less varied predictions than MHSA at k = 10.LLM-Mob assumed the most frequently visited place, 445, was the most likely next location; its generated reasons also showed hallucination problems.Source: §5.5 · p. 6

5.6 Robustness Test

  • On Geolife under k = 1 time-aware prediction, tests of temperature, prompt changes, and a different LLM showed no significant predictive-performance variation.The tests examined robustness under each of those changes.Source: §5.6 · p. 6
  • For applications favouring stable performance, the paper recommends lower temperature and consistent prompts.The authors anticipate greater uncertainty with higher temperature and larger prompt changes.Source: §5.6 · p. 6

6 Discussion

The discussion attributes LLM-Mob's performance to LLM capabilities in interpreting structured input, summarizing mobility patterns, and reasoning, combined with data formatting and prompt engineering. It also identifies efficiency, hallucination, and proprietary-model constraints.

6.1 Why does LLM-Mob perform so well?

  • The LLM performed well on tasks involving structured-input description, user activity-pattern summaries, and inference about home and workplace.The authors identify structured numerical input understanding, use of historical patterns to inform future situations, and reasoning as relevant capabilities.Source: §6.1 · p. 6
  • The authors also credit LLM-Mob's success to effective data formatting and prompt engineering refined through iterative testing.Source: §6.1 · p. 6

6.2 Limitations

  • The authors identify per-sample API inefficiency, hallucinations, and proprietary-LLM costs and performance drift as limitations.They argue these limitations motivate training open-source LLMs fine-tuned for human mobility modelling.Source: §6.2 · p. 6

7 Conclusions

LLM-Mob formats mobility data and uses context-inclusive prompts to support time-aware prediction and explanations. Experiments on two public datasets show state-of-the-art predictive performance and superior interpretability, while performance improvement and hallucination remain concerns.

7 Conclusions

  • LLM-Mob formats mobility data to account for long- and short-term dependencies and uses context-inclusive prompts to support accurate predictions and logical explanations.Table 4 marks hallucinations in test examples.Source: §7 · p. 6, Table 4 · p. 7
  • Experiments on two real-world, publicly available datasets demonstrated state-of-the-art predictive performance and superior interpretability.Table 5 covers test results from three progressive tasks, but the supplied caption does not report their values.Source: §7 · p. 7, Table 5 · p. 7
  • The authors note that predictive performance still has room for improvement and that hallucination should be addressed.Table 4 marks hallucination in blue in test examples.Source: §7 · p. 7, Table 4 · p. 7

Ethical Statement

The work used anonymised open data without personally identifiable information, while application privacy depends on the LLM provider. For travel demand forecasting, personal data are discarded after mobility patterns are aggregated to specified regions.

Ethical Statement

  • The study used anonymised open data with no personally identifiable information.Source: Ethical Statement · p. 8
  • Whether real-world applications cause privacy issues depends on the specific LLM provider; the authors expect potential risk to be minimal because proprietary LLMs abide by strict privacy policies.Source: Ethical Statement · p. 8
  • In travel demand forecasting, personal data are discarded after mobility patterns are aggregated to specified regions.Source: Ethical Statement · p. 8
Loading 2308.15197v2…