Source-linked AI summary
Who You Are Adds Nothing Detectable to Where You Go Next: Sociodemographic Conditioning in LLM Next-Location Prediction
Xin Wang, Paraic Carroll, Kerry Nice, Sachith Seneviratne, Li Zhang
TL;DR
Whether sociodemographic attributes add predictive value beyond individual mobility history remains unclear, despite their common use in LLM travel simulation. The study tests this with linked records and a paired closed-set benchmark. It finds no detectable improvement from attributes in next-location prediction, while candidate construction substantially affects measured accuracy.
Problem
Whether real sociodemographic attributes improve next-location prediction beyond an individual’s observed mobility history remains untested, despite the personal-data obligations associated with supplying them.
Source: §1 · pp. 2–3Method
The study links sociodemographic records with mobility trajectories and compares paired history-only and history-plus-attributes prompts while holding other inputs fixed in a 100-candidate benchmark.
Source: §1 · p. 3, §2.2 · p. 5, §2.3 · p. 6Results
Across paired experiments, attributes produced no detectable accuracy improvement; mismatched attributes altered rankings and reduced accuracy, and a supervised reranker reproduced the weak effect.
Source: §5 · pp. 16–17Takeaways & Limitations
Candidate construction substantially changes measured accuracy, so next-location results should be interpreted in relation to the candidate-generation procedure and information supplied.
Source: §5 · p. 17The primary result concerns closed-set ranking in one city over one week, with a single target day, 14:00 cut, and four-hour horizon.
Source: §4.3 · p. 16
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly used for individual next-location prediction, while sociodemographic conditioning is common in LLM-based travel simulation. Yet the incremental predictive value of sociodemographic attributes remains unclear. To directly test this contribution, sociodemographic records were linked with passively sensed mobility data from 5,000 Shenzhen residents to construct a closed-set benchmark in which models rank 100 candidate destinations. Each prediction instance is evaluated with and without age, gender, occupation and income, while holding mobility history, candidates and all other prompt content fixed. Results show that across four history lengths, the paired change in top-1 accuracy ranges from -0.8 to +0.5 percentage points, with no detectable gain from attributes. This result remains consistent when stay history is withheld, across alternative prediction times, in two additional LLMs and in a supervised reranker trained on the same benchmark. The null does not reflect a lack of model responsiveness to demographic information, as permuted attributes reduce LLM accuracy whereas correctly matched attributes do not improve it. A further asymmetry emerges in the reverse predictive direction, as pre-cut mobility trajectories recover income with an AUC of 0.708, while sociodemographic attributes contribute little to next-location prediction. Beyond demographic conditioning, candidate construction exerts a much larger influence on reported performance. Removing distance raises top-1 accuracy by 7.7 percentage points under proximity sampling but lowers it by 22.3 points under popularity sampling, with the reversal reproduced across all three LLMs. These results distinguish demographic association from incremental predictive usefulness and show that sampled next-location accuracy depends strongly on how candidate alternatives are constructed.
1. Introduction
LLM travel simulations commonly use sociodemographic personas, but whether real attributes improve next-location prediction beyond a person’s mobility history remains untested. The study proposes a controlled test of that incremental value and its relevance to system design and evaluation.
1. Introduction
- Whether real sociodemographic attributes improve an individual’s next-location prediction beyond observed mobility history remains untested.Prior work examines demographic group differences, synthetic personas, prediction shifts without individual histories, or profiles inferred from behaviour, leaving the incremental value of real attributes unresolved.Source: §1 · p. 2
- Demographic conditioning may entail personal-data governance obligations, stereotype risks, and greater prompt complexity, so its costs are justified only by measurable predictive value.Source: §1 · p. 3
- The study links real sociodemographic records to mobility trajectories and compares prompts with and without attributes while holding history, candidates, and other content fixed.It distinguishes representing population-level behavioural differences from improving prediction for a specific individual, and examines predictive relationships in both directions.Source: §1 · p. 3
- Candidate-set construction also affects reported accuracy and the comparability of LLM-based mobility-prediction results across studies.Source: §1 · p. 4
2. Data and Study Design
The study links Shenzhen residents’ sociodemographic records with passively sensed mobility data, then evaluates next-location ranking in a controlled, closed-set benchmark. Its paired design varies attributes and history conditions while also examining candidate construction and alternative predictors.
2.1 Linked individual attributes and mobility trajectories
- Linked records from a seven-day Shenzhen mobility observation support a 5,000-person sample with sociodemographic attributes and passively sensed trajectories.Eligibility required at least six valid days, complete attributes, and exclusion of delivery couriers and taxi drivers; operator-derived attribute classification error may attenuate observed associations. Stays were matched to nearby POIs and assigned to administrative communities.Source: §2.1 · p. 4
2.2 Prediction task and benchmark design
- Each benchmark instance ranks 100 locations at a 14:00 cut, targeting the first qualifying subsequent stay at a different location within the afternoon window.The sample contributes at most one instance per person; 1,979 people had a qualifying target, and the attribute experiments use a seeded 1,000-instance subsample. Prompts include mobility context and, in the attribute condition, age band, gender, occupation, and income.Source: §2.2 · pp. 5–6
- Proximity sampling selects nearby negatives, whereas popularity sampling uses reference-period visit frequency; the resulting candidate sets differ in spatial, visit-frequency, familiarity, and semantic composition.The candidate universe includes locations identified across the full observation week, so 6.7% of targets depend on later-observed information; excluding those instances changes paired attribute effects by no more than 0.11 percentage points. Table 1 summarizes candidate-set composition under the two protocols.Source: §2.2 · p. 6, Table 1 · p. 6
2.3 Experimental design
- A paired instance-level design compares prompts differing only in attribute inclusion, using top-1 accuracy as primary and top-5 accuracy and MRR as secondary metrics.History is truncated to K ∈ {0, 1, 2, 6} prior days; a separate K = −1 condition removes target-day stay history. Equivalence uses a ±2-percentage-point margin and a 90% bootstrap confidence interval.Source: §2.3 · pp. 6–7
- The study also tests two additional LLMs, matched-information behavioural baselines, and a supervised candidate reranker with absent, correct, or mismatched attributes.The reranker uses person-level out-of-fold splits, five seeds, and attribute-by-candidate interactions; its features exclude candidate distance. GPT-5 and Claude Opus 4.6 use fixed nested subsets and otherwise identical prompts and scoring.Source: §2.3 · pp. 7–8
3. Results
Across the tested history conditions, sociodemographic attributes add no detectable next-location accuracy; prediction instead reflects revisitation and candidate construction. The attribute null also persists in a supervised reranker, while mobility trajectories predict income more strongly in the reverse direction.
3.1 Attributes add nothing detectable, at any prior-history length
- Across four prior-history lengths, adding attributes changes top-1 accuracy by −0.8 to +0.5 percentage points, with no detectable improvement.Equivalence is concluded at K = 0, 2, and 6, but not K = 1; withholding stay history also does not uncover an attribute benefit.Source: §3.1 · pp. 8–9, Table 2 · p. 8
3.2 Prediction is dominated by individual revisitation
- Among 1,000 instances, the LLM reaches Acc@1 = 0.004 on first-time destinations, while revisit cases are more predictable through personal visit frequency.Candidate geometry also shifts model performance: when the target is not the farthest candidate, LLM accuracy rises to 0.276; when it is farthest, accuracy falls to 0.130.Source: §3.2 · p. 11, Table 4 · p. 11
3.3 The null replicates in a supervised model fitted to the benchmark
- The supervised reranker likewise shows no attribute-based top-1 improvement: the full-sample change is −0.94 percentage points.The null persists for exploration targets and lists where the target is not farthest; correctly matched attributes also provide no detectable Acc@5 advantage over permuted attributes.Source: §3.3 · p. 12, Table 5 · p. 12
3.4 Predictive usefulness is asymmetric between attributes and mobility trajectories
- Pre-cut mobility features recover above-median income with an out-of-fold AUC of 0.708, while attributes contribute little to next-location prediction.Gender, age group, and white-collar status are recovered more weakly, with AUCs from 0.545 to 0.559.Source: §3.4 · pp. 12–13, Table 6 · p. 13
3.5 Candidate construction strongly shapes measured prediction accuracy
- Under proximity sampling, removing distance raises GPT-5.5 top-1 accuracy from 0.217 to 0.293; under popularity sampling, it lowers accuracy from 0.643 to 0.420.The cue-removal reversal is reported across all three LLMs; candidate protocols also shift nearest-first heuristic accuracy from 0.018 to 0.652 while random ranking stays at 0.009.Source: §3.5 · pp. 13–14, Table 7 · p. 13, Figure 4 · p. 14
- Candidate construction can change which predictor ranks best: the frequency-and-recency heuristic beats the LLM under proximity sampling, but the ordering reverses under popularity sampling.This means reported accuracy and method comparisons depend on how alternatives are sampled.Source: §3.5 · p. 14
4. Discussion
The discussion distinguishes demographic association from incremental individual prediction: mobility history carries direct evidence of likely revisits, while attributes add little once that history is observed. It also emphasizes that candidate construction shapes measured accuracy and limits the scope of the findings.
4.1 Implications for persona-conditioned mobility modelling
- Observed mobility can reflect sociodemographic characteristics without those attributes adding comparable value to predicting an individual's next destination.The discussion links this distinction to recurring visits: personal history directly signals familiar destinations, while attributes capture broader typical behavior.Source: §4.1 · p. 15
- Demographic prompts can shift destination rankings without improving accuracy; attributes may still represent population heterogeneity, but their behavioral realism requires validation.The discussion also notes that value may differ where routines provide less information and socioeconomic constraints more directly shape travel choices.Source: §4.1 · p. 15
- The limited predictive value weakens the case for collecting attributes solely for next-location prediction, but mobility traces can still reveal socioeconomic characteristics, especially income.The discussion says privacy assessments should distinguish information needed for prediction from information inferable from the data.Source: §4.1 · p. 15
4.2 Candidate construction and comparability in next-location evaluation
- Distance can impair prediction under proximity sampling yet help under popularity sampling, so feature value depends on how candidate alternatives are generated.The sign reversal appears across all three LLMs, and distance and popularity are themselves meaningful predictors of travel behavior.Source: §4.2 · p. 15
- Reported performance across different candidate-generation procedures should not be compared directly; benchmarks should disclose sampling and compare methods on identical candidates.For LLM evaluation, the discussion also calls for specifying and holding prompt information comparable.Source: §4.2 · p. 16
4.3 Limitations
- The primary setting is one target day, a 14:00 cut, a four-hour horizon, and closed-set ranking; the findings do not establish attribute value in open-ended prediction or candidate generation.Weekday, weekend, and cut-time analyses are exploratory; the data cover one city and one week.Source: §4.3 · p. 16
- Four operator-recorded categorical attributes are coarse, and the sample may overrepresent people with stable, consistently observed mobility patterns.Household composition and vehicle access are suggested for separate study; generalization to irregularly mobile or less completely observed populations is limited.Source: §4.3 · p. 16
- The analyses do not reveal how models use distance, recover the maximum socioeconomic information in trajectories, or cover the broader space of sampling strategies.The ±2-percentage-point equivalence margin was specified after primary results were examined and should be treated as post hoc.Source: §4.3 · p. 16
5. Conclusion
The conclusion reports no detectable accuracy improvement from demographic attributes, despite evidence that models respond to demographic content. It also cautions that candidate construction changes measured performance and method comparisons.
5. Conclusion
- Adding age, gender, occupation, and income produced no detectable improvement in next-location accuracy across paired experiments.Withholding stay history did not reveal a stronger attribute effect.Source: §5 · p. 16
- Mismatched attributes altered rankings and reduced accuracy, while a supervised reranker reproduced the weak attribute effect, distinguishing model responsiveness from predictive usefulness.The conclusion notes that mobility trajectories can reveal socioeconomic characteristics, particularly income, without those attributes adding much to immediate destination prediction.Source: §5 · p. 17
- Candidate construction changes measured accuracy and can reverse the relative performance of an LLM and a behavioral heuristic, so accuracy should be interpreted with the candidate-generation procedure.The conclusion recommends reporting candidate construction and evaluating competing methods on matched candidate sets.Source: §5 · p. 17
Prompt construction, full templates and paired de-identified examples
The paired prompt design changes only the demographic attribute block between conditions. Identifiers and sensitive location details are handled separately in the prompt construction.
Prompt construction, full templates and paired de-identified examples
- Each instance was rendered under both conditions with the same history, current location, candidates, instructions, model, and decoding configuration.The attribute condition inserted a three-line block after the [User] marker without deleting or rewriting existing lines.Source: Prompt construction, full templates… · p. 19
- Individual and location identifiers were replaced by truncated salted HMAC-SHA256 values, with the salt unreleased and dates converted to relative D+ labels.Weekday and time-of-day fields were retained.Source: Prompt construction, full templates… · p. 20
- Geographic coordinates, home locations, and workplace locations were omitted, while coarse sociodemographic categories remained as the experimental intervention.Source: Prompt construction, full templates… · p. 20
Full prompt templates
The templates instruct the model to rank the full candidate list using a user message containing location history, today's stays, current location, and candidate details. The paired attribute template adds an attribute block while retaining the shared task and output instructions.
Full prompt templates
- The shared system message asks the model to rank every candidate by likelihood of being the next visited location and return only the specified JSON.Source: Full prompt templates · p. 20
- Angle-bracketed placeholders represent instance-specific values or repeated rows, and an empty history is rendered as “(none)”.Source: History-only user-message template · p. 20
- The history-only user message includes the current location and prior-day stays, with each history row carrying date, weekday, start time, duration, category, and location identifier.Source: History-only user-message template · p. 20
- The template also lists today's stays through 14:00, including start time, elapsed duration, category, and location identifier.An ongoing stay is marked when applicable.Source: History-only user-message template · p. 20
- Each of the 100 candidates is listed with its location identifier, category, and distance from the current location.Source: History-only user-message template · p. 20
- The requested output is a JSON object containing a complete ranking of all candidate IDs and one short reason sentence.Source: History-only user-message template · p. 20
- The history-plus-attributes template inserts age group, gender, occupation, and income level after the [User] marker; the shared system message is unchanged.The diff identifies the attribute heading, value line, and following blank line as the only user-message additions.Source: History-only user-message template · p. 21
Paired de-identified example 1
The paired example shows a shared mobility-and-candidate prompt context, with the history-plus-attributes version adding a demographic block and leaving the shared context unchanged.
Paired de-identified example 1
- The example uses a shared system message and a unified diff to display the condition change; the raw audit pack retains the byte-exact full-message diff.Source: Paired de-identified example 1 · p. 21
- The shared prompt supplies a current location, recent location history, and same-day stays through 14:00.The example shows an ongoing stay at the prediction cut.Source: Paired de-identified example 1 · p. 21
- The prompt presents 100 candidate locations with identifiers, categories, and distances, and requests a full ranking plus a short reason in JSON.The shown candidate list includes the first three entries and the hundredth.Source: Paired de-identified example 1 · p. 21
- Only the history-plus-attributes prompt inserts a block after [User]; no shared-context line is deleted, reordered, or rewritten.The shared context is shown once in abridged form, with omitted rows identical between conditions.Source: Paired de-identified example 1 · p. 21
- The inserted block lists age group, gender, occupation, and income level, illustrated here as 35–39, male, office worker, and level 4/5.The diff marks the block as inserted immediately after [User].Source: Paired de-identified example 1 · pp. 21–22
Paired de-identified example 2
The second paired example uses the same prompt structure: mobility and candidate context are shared, while the attributes condition adds a demographic block without changing that context.
Paired de-identified example 2
- The example uses a shared system message and a unified diff to display the condition change; the raw audit pack retains the byte-exact full-message diff.Source: Paired de-identified example 2 · p. 22
- The shared prompt supplies a current location, recent location history, and same-day stays through 14:00.The example includes prior-day entries and same-day metro-station stays.Source: Paired de-identified example 2 · p. 22
- The prompt presents 100 candidate locations with identifiers, categories, and distances, and requests a full ranking plus a short reason in JSON.The shown candidate list includes the first three entries and the hundredth.Source: Paired de-identified example 2 · p. 22
- Only the history-plus-attributes prompt inserts a block after [User]; no shared-context line is deleted, reordered, or rewritten.The shared context is shown once in abridged form, with omitted rows identical between conditions.Source: Paired de-identified example 2 · p. 22
- The inserted block lists age group, gender, occupation, and income level, illustrated here as 35–39, male, office worker, and level 4/5.The diff marks the block as inserted immediately after [User].Source: Paired de-identified example 2 · p. 22