PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Models for Travel Behavior Prediction

Baichuan Mo, Hanyong Xu, Ruoyun Ma, Jung-Hoon Cho, Dingyi Zhuang, Xiaotong Guo, Jinhua Zhao

arXiv:2312.00819v2cs.LGcs.AIcs.CL

TL;DR

The paper asks whether LLMs can predict travel behavior, where systematic evidence is limited for tasks such as mode choice and trip purpose. It tests zero-shot prompting and LLM-generated embeddings paired with supervised models. The approaches perform comparably to classical models, with potential as flexible, data-efficient alternatives.

  • Problem

    Systematic evidence on LLM prediction of travel behavior is limited, particularly for mode choice and trip purpose, despite the importance of travel prediction in transportation planning.

    Source: §1 · p. 1, §2.2 · p. 3

  • Method

    The study tests zero-shot prompts incorporating travel tasks, traveler attributes, and domain knowledge, alongside LLM-generated text embeddings used as features for supervised prediction.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Both LLM frameworks achieve performance comparable to, and in some cases competitive with, multinomial logit, random forest, and neural networks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings suggest LLMs may serve as flexible and data-efficient alternatives for travel behavior prediction.

    Source: §1 · p. 2

    This initial exploration needs broader, more diverse datasets, repeated trials, and evaluations of additional travel behavior tasks.

    Source: §5 · p. 13

Abstract

from arXiv · show

Travel behavior prediction is a core problem in transportation demand management and is traditionally addressed using numerical models calibrated on observed data. With recent advances in large language models (LLMs), new opportunities have emerged to model human decision-making through natural language reasoning. This study explores the use of LLMs for travel behavior prediction through two complementary frameworks. The first framework employs a zero-shot prompting strategy, where the prediction task, traveler attributes, and relevant domain knowledge are described in text, enabling the LLM to directly generate predictions without task-specific training data. The second framework uses LLM-generated text embeddings as high-level representations of travel scenarios, which are then combined with conventional supervised learning models to support prediction in small-sample settings. Empirical results show that both approaches achieve performance comparable to, and in some cases competitive with, classical models such as multinomial logit, random forest, and neural networks. These findings suggest that LLMs offer a flexible and data-efficient alternative for travel behavior prediction.

1. Introduction

Travel behavior prediction matters to transportation planning, but whether LLMs can predict it remains unclear. The study examines two LLM-based approaches and reports performance comparable to classical models in representative tasks.

1. Introduction

  • Travel mode choice concerns how people select their means of travel, and accurate predictions can inform transportation agencies and individual travelers.For agencies, predictions can support transit services, short-term forecasts, infrastructure investments, policy, and land-use planning; for individuals, they can inform map-app decisions.Source: §1 · p. 1
  • LLMs’ ability to predict travel behavior remains unclear, despite prior demonstrations of their mobility-prediction capabilities.The study focuses on mode choice and trip purpose as representative tasks.Source: §1 · p. 2
  • The study proposes zero-shot prompting and an embedding-based framework that feeds LLM-generated representations into supervised models for small-sample prediction.The zero-shot framework describes the task, traveler attributes, and domain knowledge in prompts; it does not require task-specific training data.Source: §1 · p. 2
  • Both frameworks achieve performance comparable to, and sometimes competitive with, multinomial logit, random forest, and neural networks.The paper also describes the approaches as flexible and data-efficient tools for travel behavior prediction.Source: §1 · p. 2

2. Related work

The review covers LLM capabilities and limitations, then surveys their use in human mobility prediction. It identifies limited systematic study of zero-shot LLM prediction for travel behavior, particularly mode choice and trip purpose.

2.1. Recent developments in large language models

  • LLMs are described as capable of complex language tasks, while prompting approaches such as in-context learning, chain-of-thought, and plan-and-solve guide their responses.The review also notes that LLM performance often depends on prompt engineering.Source: §2.1 · pp. 2–3
  • LLM limitations include hallucinations, limited access to recent knowledge, and reasoning that may not align with answers.The review notes that generated text can conflict with itself or be unverifiable.Source: §2.1 · p. 3

2.2. Large language model as a predictor in human mobility

  • Transportation applications of LLMs include traffic, human-mobility, and demand forecasting, as well as data imputation.The review places these uses among broader applications of LLMs in transportation.Source: §2.2 · p. 3
  • Prior mobility studies have used language descriptions, encodings, embeddings, prompts, and semantic sequence data for trajectory or time-series prediction.The review reports that Wang et al. (2023b) found GPT predictions outperformed LSTM and MHSA models for next-location prediction.Source: §2.2 · p. 3
  • Prior work also applies LLM planning to trip planning through multiple information sources and stepwise decomposition.The review describes these approaches as relatively successful.Source: §2.2 · p. 3
  • The review identifies little systematic research on zero-shot LLM prediction of travel behavior, especially mode choice and travel purpose.Existing travel-choice studies had examined historical personalized information, persona loading, and few-shot approaches.Source: §2.2 · p. 3

3.1. Preliminaries

The preliminaries describe LLM pretraining and text generation, then introduce hidden text embeddings as features for downstream supervised prediction.

3.1. Preliminaries

  • LLMs are presented as transformer-based neural networks that process and generate text using self-attention to model long-range dependencies.The passage also describes model scale as having grown to hundreds of millions or billions of parameters.Source: §3.1 · p. 4
  • Pretraining exposes a model to large text corpora to learn language properties, with next-word prediction as the primary objective in the described GPT framework.The training corpus is denoted U, and the passage identifies w_i^(u) as the i-th token in sequence u.Source: §3.1 · p. 4
  • After training, the model generates an answer sequence token by token until it reaches the [End] token.In real-world implementation, [End] is added to the input and is not displayed at the end of the output.Source: §3.1 · p. 4
  • LLM text embeddings provide hidden-vector features for downstream supervised learning, with their dimension depending on the model architecture.The paper suggests these representations could improve prediction with small sample sizes.Source: §3.1 · p. 4

3.2. Conceptual framework

The framework uses LLMs for travel behavior prediction through prompts, with outputs that can be either generated predictions and reasons or embeddings used by a supervised model.

3.2. Conceptual framework

  • The framework proposes two ways to incorporate LLMs into travel behavior prediction.Travel mode choice and trip-purpose prediction serve as illustrative tasks.Source: §3.2 · p. 5
  • Prompts combine task descriptions, structural and descriptive data, and domain-knowledge reasoning guides.The framework feeds the prompt to an LLM to generate predictions and associated reasons, without a supervised training step.Source: §3.2 · p. 5, Figure 1 · p. 5
  • The alternative framework extracts an LLM hidden vector and uses it as features for a supervised model, such as multinomial logit.The trained model predicts the travel behavior.Source: §3.2 · p. 5

3.3. Prompt design

The prompt design combines task information and traveler context with domain knowledge to guide LLM predictions; the prompt specifies the requested outputs and their format.

3.3.1. Task description

  • Task descriptions specify the prediction options and the information that will be provided to the LLM.The task may concern mode choice, travel purpose, departure time, duration, destination, or other travel behavior.Source: §3.3.1 · p. 5, Figure 2 · p. 6

3.3.2. Structural data

  • Travel characteristics such as origin, destination, travel time, and cost are organized in a dictionary format.The paper describes these characteristics as major determinants of travel behavior.Source: §3.3.2 · p. 6, Figure 3 · p. 6

3.3.3. Descriptive data

  • Individual attributes are described in text rather than represented with the dummy binary variables typical of mathematical models.Source: §3.3.3 · p. 6, Figure 4 · p. 6

3.3.4. Guide of thinking with domain knowledge

  • Prompts guide reasoning with domain knowledge and human common sense, using strategies such as Chain-of-Thought and Plan-to-Solve.The paper calls this guidance the most important part of enabling prediction without training data.Source: §3.3.4 · p. 6, Figure 5 · p. 7
  • For numerical comparisons, prompt guidance identifies the mode with the lowest travel time or cost and states relative differences.The authors report that this tends to be effective for arithmetic reasoning in this context.Source: §3.3.4 · p. 7

3.3.5. Interpretation and output

  • The requested output includes predictions and supporting reasons, with a specified format for easier extraction.The authors say reasons aid prompt diagnosis and interpretability and can improve LLM reasoning.Source: §3.3.5 · p. 7, Figure 6 · p. 7

3.3.6. Summary

  • The complete prompt is reordered from its component sequence and includes no training-data information, making it zero-shot.Source: §3.3.6 · p. 7, Figure 7 · p. 8

3.4. Embedding as features

LLM embeddings represent traveler, trip, or travel-context information for downstream prediction, separating representation learning from the supervised prediction model.

3.4. Embedding as features

  • Each sample's embedding h_n serves as a compact representation of behavioral features, preferences, and context.A sample may represent a traveler, trip record, or textual description of travel context.Source: §3.4 · p. 8
  • A supervised model uses h_n to predict the travel behavior label y_n and is trained to minimize empirical loss over the training dataset.For classification, the sample-wise loss is typically cross-entropy; for regression, it may be mean squared error or another appropriate metric.Source: §3.4 · p. 8
  • Separating LLM representation learning from prediction allows downstream models to be selected for task requirements and computational considerations.Examples include logistic regression, random forests, gradient boosting machines, and neural networks.Source: §3.4 · p. 8

4. Case study

The case studies evaluate travel mode choice and trip purpose prediction with balanced samples, several conventional baselines, and LLM approaches. Large-sample supervised models perform best, while zero-shot predictions and text embeddings show value in low-data settings; prompt sensitivity differs by model.

4.1. Data

  • Mode choice uses the Swissmetro stated-preference survey, and trip purpose uses the 2017 US National Household Travel Survey.Swissmetro data cover train, car, and Swissmetro choices; NHTS covers working, social, recreation, and other trip purposes.Source: §4.1 · p. 9
  • Balanced sampling selects 1,000 training entries, 10 benchmark-training entries, and 200 test entries, averaging results across five random samples.The label proportions are kept the same across samples.Source: §4.1 · p. 9

4.4.1. Prediction performance

  • Large-sample supervised models achieve the best overall performance, while their accuracy and weighted F1-score deteriorate substantially with small training sets.The reported metrics are prediction accuracy and weighted F1-score.Source: Table 3 · p. 10, §4.4.1 · p. 10
  • Zero-shot LLM predictions without training samples can outperform supervised models trained on limited data.Supervised learning uses training sets of 1,000 or 10 samples; zero-shot LLMs use no training samples.Source: §4.4.1 · p. 10
  • With small training datasets, LLM-generated text embeddings consistently improve supervised-model performance over manually engineered features.The authors attribute this to semantic information that is difficult to encode through traditional feature design.Source: §4.4.1 · p. 11

4.4.2. Interpretability

  • The LLM outputs include plausible explanations for correct and incorrect predictions, but also reasoning errors and a hallucinated time-saving preference unsupported by inputs.One example gives a reasonable explanation for a wrong car prediction; another cites a missing train annual pass as a reason to choose train over Swissmetro.Source: Table 4 · p. 11, §4.4.2 · p. 12

4.4.3. Ablation studies

  • In Swissmetro model-choice ablations, excluding the guide of thinking led LLM-GPT-3.5's accuracy to drop by 5.8%, while LLM-GPT-4's accuracy decreased by less than 1% across all ablation settings.Source: Table 5 · p. 12, §4.4.3 · pp. 12–13
  • For GPT-3.5, descriptive rather than structured inputs decrease performance by 4.4%, while removing domain knowledge decreases it by 0.5%.The authors report that GPT-4 remains largely stable across the tested ablation settings.Source: Table 5 · p. 12, §4.4.3 · p. 13

5. Conclusion and future work

The study finds that zero-shot LLM predictions and LLM-generated embeddings can complement conventional travel prediction models, especially with scarce labeled data. It also reports explanation errors and model-specific prompt sensitivity, and identifies broader evaluation and reliability work as future needs.

5. Conclusion and future work

  • Across mode-choice and trip-purpose prediction, zero-shot LLMs compete without training data and outperform supervised models trained on very small samples.Supervised models perform best when large labeled datasets are available, but their effectiveness degrades substantially in low-data regimes.Source: §5 · p. 13
  • LLM-generated embeddings significantly improve supervised-model accuracy and F1 scores under data-scarce conditions.The authors associate this improvement with high-level semantic information that manual feature engineering is difficult to capture.Source: §5 · p. 13
  • Natural-language explanations can be reasonable even for incorrect predictions, but observed reasoning errors and hallucinations warrant caution in decision-critical applications.GPT-3.5 is more sensitive to prompt design, while GPT-4 is reported as robust to prompt variations.Source: §5 · p. 13
  • The authors call for larger and more diverse datasets, repeated trials, more travel tasks, systematic prompt studies, and broader model comparisons.They also propose exploring few-shot and in-context learning and methods to reduce hallucinations and improve explanation faithfulness.Source: §5 · p. 13

Appendix A. Prompts

Appendix A presents the final prompts used for travel mode and trip purpose predictions. Each prompt includes an example, with sample-dependent parts underscored.

Appendix A. Prompts

Loading 2312.00819v2…