PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating LLM-Generated Preference Distributions

Fan Huang, Minsuk Kim, C. Tyler Diggans, Filippo Radicchi

arXiv:2610.01000v1cs.AI

TL;DR

The paper addresses the limited evidence on the structure and reliability of LLM-generated preference distributions. It analyzes repeated samples of elicited distributions across models and choice domains, finding rapid stabilization of dominant outcomes alongside substantial discordance across models. The results point to model choice as a primary factor shaping these distributions, while the study does not establish their correspondence to human preferences.

  • Problem

    The distributional structure of LLM outputs under repeated sampling, model scaling, and prompt changes remains underexplored despite growing use of LLMs as probabilistic generators.

    Source: §1 · p. 1

  • Method

    The paper elicits explicit probability distributions over discrete choices and analyzes repeated samples across nine open-weight models, three families, and three preference domains.

    Source: §1 · p. 2

  • Results

    Across the studied models, dominant modes stabilize rapidly, but distributions are discordant across model families and scales, with prompt shifts smaller than Llama scale differences.

    Source: §1 · p. 2, §7 · pp. 9–10

  • Takeaways & Limitations

    Model selection should be treated as a primary experimental variable, and a few independent samples can identify dominant outcomes but not characterize tail uncertainty.

    Source: §7 · p. 10

    The study does not benchmark against ground-truth human preferences, so correspondence between elicited priors and real-world choice behavior remains unstudied.

    Source: §6 · p. 9

Abstract

from arXiv · show

Large Language Models (LLMs) are increasingly used as probabilistic generators for simulation, synthetic data generation, and decision support in settings where real-world data are unavailable. Yet, the structure and reliability of the distributions they produce remain understudied. Here, we systematically analyze LLM-generated distributions of preferences for air travel, restaurants, and consumer products. Encouragingly, all models considered in our analysis exhibit self-coherence, with the most probable outcomes stabilizing rapidly under repeated sampling. At the same time, we observe substantial discordance across both model families and scales, with little consensus even among their most probable outcomes. These patterns hold across nine open-weight models, three choice domains, and show robustness under temperature changes, greedy decoding, and perturbations of prompt and ordering. Our findings indicate that outcomes are influenced more by the choice of model than by the wording of the prompt, challenging the common assumption that sufficiently capable LLMs produce similar preference distributions when used as stand-ins for survey respondents.

1 Introduction

The introduction argues that LLMs are increasingly treated as probabilistic stand-ins, although the structure of their full output distributions remains underexplored. It frames the study around whether preference distributions stabilize, vary by model scale, respond to prompts, and generalize across domains and model families.

1 Introduction

  • An implicit world prior is an externally observable, reproducible mapping from a natural-language scenario to an outcome distribution, distinct from a full world model.For downstream simulation, the distribution should be stable under repeated sampling, internally structured, and boundedly responsive to controlled prompt changes.Source: §1 · p. 1
  • LLM distributional behavior remains underexplored because evaluations often assess single-output accuracy, calibration, or aggregate performance rather than repeated-sampling distributions and prompt-induced shifts.The study focuses on preferences for air travel, restaurants, and consumer products.Source: §1 · p. 1
  • The study asks whether repeated sampling stabilizes modes before tails, whether model sizes share priors, how prompts shift distributions, and whether patterns generalize across domains and families.These questions target both within-family scale differences and prompt perturbations.Source: §1 · p. 2
  • Across nine open-weight models and three domains, modes consolidate quickly, while Llama scales show JS divergence of 0.36–0.50 and top-5 agreement below 0.13, exceeding prompt shifts of 0.13–0.33.Discordance is milder but persists in Qwen and Mistral; findings are validated across temperature, decoding, prompt, ordering, and choice-set-size variations.Source: §1 · p. 2

2 Related Work

Related work examines LLMs as behavioral simulators, studies calibration and uncertainty in individual outputs, and documents prompt sensitivity. The paper positions its contribution as characterizing full output distributions, including their stability, convergence, and model sensitivity.

2.1 LLMs for Simulation and Prediction

  • Prior work finds both subgroup-aligned response distributions and substantial opinion misalignment, while leaving full-distribution stability, convergence, and model sensitivity largely uncharacterized.Argyle et al. (2023) report correlations with matching human subgroups; Santurkar et al. (2023) and Tjuatja et al. (2024) report misalignment or failures to reflect human-like response biases.Source: §2.1 · p. 2

2.2 Probabilistic Calibration and Uncertainty

2.3 Prompt Sensitivity and Robustness

2.4 Mode Collapse and Numeric Calibration

  • Related work covers generation diversity loss during fine-tuning, reluctance to express uncertainty, poor numeric calibration, and evaluation beyond isolated test cases.The passage cites Janus (2022), O’Mahony et al. (2024), Zhou et al. (2024), Lovering et al. (2025), and Lum et al. (2025).Source: §2.4 · pp. 2–3

3 Methods

The methods elicit explicit probability distributions over discrete choices and analyze them as repeated samples under controlled prompts. The framework separates dominant modes from low-probability tails and evaluates consistency, convergence, variability, and prompt or scale differences.

3 Methods

  • The framework elicits explicit probability distributions over large discrete choice sets, using air travel as the primary domain and restaurants and products for cross-domain validation.Air travel uses 300 contiguous-US commercial airports; the framework is described as domain-agnostic.Source: §3 · p. 3

3.1 Problem Formulation

  • An implicit world prior is modeled as p(y | x), a distribution over finite outcomes conditioned on a natural-language scenario, whose structure, stability, and sensitivity are analyzed.The prompt may describe a departure airport and traveler persona.Source: §3.1 · p. 3

3.2 Elicitation Method Validation

  • Verbalized elicitation asks for probabilities over all options in one query; it supports reproducibility and structural-consistency analysis, while logit inspection and repeated outcome sampling were infeasible at study scale.Hu and Levy (2023) argue that direct logit readout is more faithful for probability elicitation, but their and Lovering et al. (2025)’s focus is calibration to ground truth.Source: §3.2 · p. 3

3.3 Iterative Sampling Protocol

  • Each prompt is sampled through up to 20 independent calls at temperature 0.2, with cumulative metrics invariant to sample order by construction.The sample cap was selected to observe convergence of mean distributions across model sizes while keeping invocation counts reasonable; airport-list ordering is checked separately.Source: §3.3 · p. 3

3.4 Mode and Tail Regions

  • Outcomes are partitioned into the k highest-probability modes and their complementary tail, with k=5 supported by observed probability gaps and sensitivity checks.The natural mode–tail boundary falls at rank 5 or earlier in 99% of distributions across 300 departure contexts; results are preserved for cutoffs k≥5.Source: §3.4 · p. 3

3.5 Evaluation Metrics

  • The evaluation tracks support growth, entropy, tail mass, cumulative unique outcomes, convergence of top-k sets, and distributional consistency under repeated sampling.Unique@R is the cumulative union of observed supports; final support counts outcomes above 0.1% probability in the converged distribution.Source: §3.5 · p. 4
  • JS divergence compares full, mode, and tail regions, while cross-scale comparisons pair divergence with top-k agreement and prompt-condition comparisons use divergence and Jaccard@k.The regional JS measures are complementary views, not an additive decomposition; Stable@ is reported only as a descriptive complement to top-k stabilization.Source: §3.5 · p. 4

4 Experiments

Experiments primarily elicit air-travel preferences from structured prompts, with extensions across models, temperatures, and domains. The design uses repeated sampling and subsampling to assess convergence and quantify uncertainty in reported metrics.

4 Experiments

  • Air-travel prompts specify a persona, departure airport, 300 destination airports, and probability-output instructions; 20 unweighted age–education combinations define personas.The airport choice set represents the contiguous-US network; the study also validates across restaurant and product choices.Source: §4 · p. 4
  • Experiments primarily use Llama-3.3-70B at temperature 0.2, extend to nine models across three families, and replicate under greedy decoding.Temperature 0.2 was validated across ten temperatures and six settings; malformed outputs are normalized to airport codes where possible and otherwise discarded.Source: §4 · p. 5
  • Bootstrap analyses repeatedly subsample up to 20 independent samples without replacement and report percentile confidence intervals for metric estimates.At the full sample count the interval is zero by construction, so curves’ informative region is at small sample sizes rather than the right endpoint.Source: §4 · p. 5
  • Table estimates report means with half-widths of 95% bootstrap confidence intervals, resampling personas or pairwise distribution comparisons as appropriate.Single-context entries without defined intervals are flagged in captions.Source: §4 · p. 5

5 Results

Across the results, LLM-generated distributions show rapidly stabilizing modes alongside substantial model-scale and family discordance. Prompt shifts are bounded and structured, the patterns extend across domains, and robustness checks support the findings.

5.1 Rapid Stabilization of Modes of the Distributions

  • High-probability outcomes stabilize across independent samples: mean pairwise Jaccard@5 is 0.72, and 36 of 74 sample pairs have identical top-5 sets.The cumulative support stabilizes within a few samples, while Spearman rank correlation exceeds 0.95 at stabilization points.Source: §5.1 · p. 6, Table 1 · p. 6

5.2 Post-Convergence Variability is Tail-Dominated

  • After convergence, tail divergence exceeds mode divergence by 2.1–2.3×, with 36.7–40.2% of probability mass outside the top five.The authors note that renormalizing noisy tail outcomes magnifies their relative divergence; greedy decoding preserves the modes-first pattern.Source: §5.2 · p. 6, Table 2 · p. 6

5.3 Distributional Discordance Across Scales and Families

  • JS divergence is 0.36–0.50 and agreement@5 is 0.08–0.12 across Llama sizes, showing substantial discordance rather than simple scale refinement.Figure 1 also shows distinct distributional signatures across the three sizes; Table 3 reports the pairwise metrics.Source: §5.3 · p. 7, Table 3 · p. 7, Figure 1 · p. 7
  • Across Qwen, Mistral, and Llama, scale discordance persists; matched single-context Llama JS divergence is 0.17–0.33, comparable in magnitude to the other families.The Qwen and Mistral range is 0.14–0.28; their original estimates are single-context, unlike the Llama 20-persona means.Source: §5.3 · pp. 7–8, Table 4 · p. 8

5.4 Prompt Perturbations are Bounded and Structured

  • Prompt perturbations yield different-sized shifts: age has JS 0.33, context 0.31, education 0.26, and persona 0.13; seasonal divergence is 0.098.Seasonal effects are smaller at large hubs than at regional airports; perturbation shifts are smaller than changing model scale.Source: §5.4 · p. 8, Table 5 · p. 8

5.5 Cross-domain generalization

  • Modes-first convergence holds across travel, Yelp, and Amazon, with full convergence by iterations 8, 11, and 17, respectively.Top-10 probability mass is 79% for Amazon, 75% for travel, and 49% for Yelp; the authors relate the differences to choice-set utility structure.Source: §5.5 · p. 8

5.6 Robustness of Findings

  • Temperature affects maximum probability, entropy, and top-5 mass in all six settings; top-5 ranking stability is 0.93 at T=0.2 versus 0.81 at T=1.0.All main findings persist under greedy decoding (T=0).Source: §5.6 · pp. 8–9
  • Across ordering, prompt paraphrases, and origin airports, all metrics have CV below 21%; model scale contributes the largest variance share in distributional spread (support ICC = 0.46).Airport and persona contribute little systematic between-condition variance; model scale has the largest ICC on every metric except top-5 mass.Source: §5.6 · p. 9

6 Discussion and Limitations

Model choice shapes structurally different priors across families and scales, while prompt effects are bounded; these conclusions are limited by the study’s settings and unresolved scale-dependent comparisons.

6 Discussion and Limitations

  • Across three families and domains, different models encode structurally different priors rather than refinements of one another, making model choice the dominant lever overall.The comparison includes both different families and scales within a family.Source: §6 · p. 9
  • The analysis covers enumerable discrete choice spaces of roughly hundreds of outcomes, and its patterns may not transfer to open-ended generation.Only open-weight models were evaluated; whether the same dynamics hold for closed-source systems remains open.Source: §6 · p. 9
  • Prompts were English-only, and without a ground-truth human-preference benchmark, correspondence between the elicited priors and real-world choices remains unstudied.The paper frames internal consistency and structure as the operative questions where a reference distribution is unobservable.Source: §6 · p. 9
  • The model-versus-prompt margin depends on scale gap: divergences of 0.474 and 0.501 from 8B to larger checkpoints exceed measured prompt shifts, while 0.365 for 70B versus 405B overlaps the largest age (0.330) and context (0.310) effects.The 8B comparisons have non-overlapping confidence intervals; the adjacent-large-checkpoint and largest-prompt-effect intervals overlap and are unresolved at the persona sample size.Source: §6 · p. 9
  • The observed regularities characterize external behavior but do not identify a specific internal mechanism.Multiple internal processes may produce similar behavior.Source: §6 · p. 9

7 Conclusion and Future Work

Across nine open-weight models, the elicited distributions show modes-first stabilization alongside persistent tail variability and disagreement across models. The authors recommend treating model selection as a primary variable and call for validation and broader extensions.

7 Conclusion and Future Work

  • Top-k outcomes stabilize within a few samples while tail variability persists, and modes-first dynamics replicate across three choice domains and, for travel, all three model families.The conclusion describes this distributional anatomy across nine models, three families, and three preference domains.Source: §7 · pp. 9–10
  • For Llama, prompt perturbations induce systematic but bounded shifts below the effect of changing scale.The modes-first pattern also replicates across the three choice domains and, in travel, across all three model families.Source: §7 · p. 10
  • Temperature amplifies or dampens tail variability but does not create it; most of the tail remains under greedy decoding (T=0).Source: §7 · p. 10
  • The outcomes of models differ strongly across both families and scales.Source: §7 · p. 10
  • A small number of independent samples can identify dominant outcomes but not characterize tail uncertainty, so model selection should be a primary experimental variable.The authors recommend reporting each LLM-derived distribution with its producing model; future work includes human-preference validation and extensions to closed-source models, other architectures, and non-English settings.Source: §7 · p. 10

Ethical Considerations

The authors caution against normative or policy use and recommend validation and domain expertise for downstream applications. They also flag risks of misinterpreting demographic conditioning and the costs of large-scale inference.

Ethical Considerations

  • The analyses are not intended to support normative or policy decisions; downstream uses should include real-world validation and appropriate domain expertise.Source: Ethical Considerations · p. 10
  • Because model choice substantially influences generated distributions, applied uses should make model selection transparent and include sensitivity analyses.Source: Ethical Considerations · p. 10
  • The experiments use synthetic data without personal information, but the authors caution against misinterpreting demographically conditioned outputs; large-scale inference also has computational and environmental costs.They encourage practitioners to consider those costs when planning experiments.Source: Ethical Considerations · p. 10

A Elicitation Method Validation: Full Comparison

This section validates preference-elicitation approaches by comparing verbalized elicitation with repeated single-choice sampling and defining the evaluation metrics and thresholds. It also documents the cross-domain convergence comparisons, dataset construction, prompt and experiment design, and parsing challenges for smaller models.

A Elicitation Method Validation: Full Comparison

  • At T=0.2, 300 single-choice calls produced one destination with 0 bits of entropy; at T=1.0, they produced two with 0.18 bits, versus 34–35 destinations and 3.97–4.25 bits for verbalized elicitation.The JS divergence between single-choice and verbalized distributions was 0.64 at T=0.2 and 0.61 at T=1.0; one verbalized call yielded 35 destinations, compared with two from 300 single-choice calls at T=1.0.Source: A Elicitation Method Validation: Full… · pp. 12–13, Table 6 · p. 13

B.1 Distributional Consistency Metrics

  • Distributional consistency is assessed with JS divergence, top-k Jaccard overlap, and Spearman rank correlation; lower JS indicates greater similarity, while higher overlap and correlation indicate greater stability.JS divergence is symmetric and bounded by [0, ln 2] with natural logarithms, except Table 6 uses base-2 logarithms; JS distance is the square root of divergence and is not directly comparable to it.Source: §B.1 · p. 13

B.2 Tail Dynamics Metrics

  • Tail dynamics are characterized by support at ε=0.001, Shannon entropy, and tail mass, the probability allocated outside the top-k mode set.Higher support indicates broader coverage; higher entropy indicates a less concentrated distribution, while higher tail mass indicates more probability on low-probability alternatives.Source: §B.2 · pp. 13–14

B.3 Mode–Tail Divergence Analysis

  • Mode–tail analysis computes total, mode-region, and tail-region JS divergences independently, so the regional measures are complementary views rather than an additive decomposition.Tail divergence is expected to exceed total divergence because renormalization magnifies small differences among low-probability outcomes; mode divergence can approach zero while tail divergence remains elevated.Source: §B.3 · p. 14

B.4 Prompt Sensitivity Analysis; B.5 Scale Discordance Analysis

  • Prompt sensitivity is measured as mean JS divergence across comparison units, while scale discordance uses cross-model JS divergence and agreement@k for identical top-k sets.The perturbation unit varies by comparison: departure context, persona pairs differing in one demographic attribute, or a persona at regional versus hub airports.Source: §B.4 · p. 14, §B.5 · p. 14

B.6 Justification for k=5 Mode–Tail Threshold

  • The largest adjacent-probability gap fell at rank 5 or earlier in 99% of 300 departure contexts, grounding the k=5 mode–tail cutoff.Across k≥5, tail divergence exceeded mode divergence in both post-convergence phases; post-convergence mean Jaccard@k stayed between 0.72 and 0.77, while stabilization points shifted later as k grew.Source: §B.6 · p. 15, Table 8 · p. 16

B.7 Threshold-Insensitivity of the Stabilization Point

  • Across all four primary contexts, the k=5 stabilization point was unchanged for every Jaccard threshold τ between 0.67 and 1.0, making the chosen 0.9 threshold immaterial.Across 10 tested temperatures, tail divergence exceeded mode divergence; Jaccard@10 stayed between 0.81 and 0.91, Jaccard@5 declined from 0.93 to 0.81, and top-5 mass stayed between 0.49 and 0.57.Source: §B.7 · p. 15, §B.6 · p. 15

C Experimental Configuration Details; C.1 Persona Configuration; C.2 Prompt Templates

  • The persona grid contains 20 age–education combinations, and prompts combine a persona, choice set, question, and output-format instruction.The persona combinations are all included as conditions without ACS population weighting; the templates cover persona-conditioned travel and no-persona, seasonal, restaurant, and product-choice variants in the supplied excerpts.Source: §C.1 · pp. 16–17, §C.2 · p. 17

C.2 Prompt Templates

  • Product prompts list 300 options and request purchase probabilities from 0 to 1 that sum to 1.0, formatted as a JSON array.The question asks for each product’s probability of purchase in the next month.Source: §C.2 · p. 17

C.3 Research Question Details

  • Experiments assess convergence across four departure contexts, prompt sensitivity to demographics, season, and geography, and scale discordance across model families and domains.The seasonal comparison uses four airports and 160 total runs; cross-domain replication uses 300 Yelp restaurants and 300 Amazon products.Source: §C.3 · p. 18

D.1 Yelp Restaurant Dataset

  • The Yelp choice set contains 300 Indianapolis restaurants across 54 cuisine categories, with records including names, cuisine, ratings, review counts, and location attributes.Restaurants were extracted from the Yelp Open Dataset using location, dining-category, and name-quality criteria.Source: §D.1 · p. 18

D.2 Amazon Product Dataset

  • Amazon sampling yields 300 Home & Kitchen products balanced across 15 subcategories, using embedding-based clustering and filtering out near-duplicates above 0.90 cosine similarity.Twenty products were selected per subcategory; titles and brands were embedded, and cluster representatives were sampled.Source: §D.2 · pp. 18–19

D.3 Domain Comparison

  • The three domains differ in key characteristics, enabling tests of whether convergence, scale discordance, and prompt-sensitivity patterns generalize beyond travel.The section presents a domain-characteristics comparison in Table 9.Source: §D.3 · p. 19, Table 9 · p. 19

D.4 Cross-Domain Convergence Comparison

  • Table 10 compares convergence metrics for travel, Yelp, and Amazon using single-context, 20-iteration runs, with JS convergence defined as the first iteration below 0.05.The table also reports top-10 mass at a common k=10 cutoff; the broader travel result across 300 departure contexts is reported separately.Source: Table 10 · p. 19, §D.4 · p. 19

E Parsing Methodology for Smaller Models

  • Smaller models require specialized parsing strategies: Llama-3.1-8B had significant failures under the original JSON-bracket-only parser, and an unrectified baseline is reported for transparency.The section documents parsing challenges and rectification methodology for smaller models.Source: §E.1 · p. 19, E Parsing Methodology for Smaller Models · p. 19

E.2 Failure Case Analysis

Smaller-model outputs often failed the requested format, motivating a three-tier parser; without rectification, only a minority of the smaller model’s results could be analyzed. The appendices also show structured destination preferences and convergence patterns.

E.3 Three-Tier Parsing Algorithm

  • To address parsing failures, the authors implemented a three-tier approach: bracket extraction (original method), regex JSON extraction (new), and natural-language extraction (new); max_tokens increased from 1024 to 4096 to prevent truncation of verbose responses.Source: §E.3 · p. 20

E.4 Original Results (Without Rectification)

  • Only 3/20 personas (15%) and 6/20 iterations (30%) parsed successfully without rectification, leaving convergence analysis impossible.The 17 remaining personas returned malformed JSON; the six parsed iterations yielded 63 records.Source: §E.4 · p. 20

F Detailed Experimental Results

  • The complete airport probability matrix shows geographic structure, with major hubs appearing as consistently high-probability modes.Only 185 of the 300 offered airports received non-zero probability in at least one context.Source: F Detailed Experimental Results · p. 20, Figure 3 · p. 21

G Convergence and Stability Analysis

  • Cross-domain and regional visualizations show rapid stabilization of modes followed by gradual tail expansion.The cross-domain convergence patterns are visualized for Yelp and Amazon, while the regional heatmap tracks 32 destinations across 20 iterations.Source: G Convergence and Stability Analysis · p. 21

G.1 Extended Stability Analysis

The extended analyses report model-scale differences in stability and distributional diversity, including non-monotonic scaling. Cross-family comparisons also show family-dependent diversity patterns, while output normalization and collection adaptations address comparability and response failures.

G.1 Extended Stability Analysis

  • The 8B model has the highest run-to-run divergence (mean JS = 0.483 vs. 0.171 and 0.152 for 70B and 405B), while pairwise divergence decreases and top-k overlap increases with more samples.The 8B model also has a larger mode contribution to total divergence.Source: §G.1 · p. 21

H Model Size Comparison: Full Heatmaps

H.1 Non-Monotonic Scaling of Distributional Diversity

  • The 8B model yields 100 final destinations, versus 32 for 70B and 54 for 405B; its maximum probability is 9.0%, compared with 17.1% and 17.4%.The 70B model truncates low-probability alternatives more aggressively, while 405B retains a longer tail.Source: §H.1 · p. 22, Figure 7 · p. 23

I Cross-Family Methodology

  • The cross-family evaluation covers nine Llama, Qwen, and Mistral models; destination normalization restores shared support for comparison.Without normalization, differing output formats produced zero common destinations and maximum JS divergence (0.693); after normalization, typical common counts were 15–30.Source: §I.1 · p. 23, §I.3 · pp. 24–25

I.2 Per-Scale Distributional Signatures: Qwen and Mistral

  • Across Qwen and Mistral, the most diffuse scale differs by family: the largest Qwen model, 235B, versus the smallest Mistral model, 7B.Figures 10 and 11 show distinct operating points rather than a simple refinement across scales.Source: §I.2 · p. 24, Figure 10 · p. 24, Figure 11 · p. 25

I.4 Mistral-Small-24B Behavioral Adaptation

  • Mistral-Small-24B returned valid responses in only 11.2% of attempts, prompting a research-context prefix and retries to obtain 20 valid samples.Clarification behavior was not observed in persona-conditioned experiments.Source: §I.4 · p. 25

I.5 Experimental Parameters

  • Cross-family experiments used T=0.2, 20 iterations, Indianapolis, and a 300-airport choice set, with 20 valid samples collected per model.Mistral-Small-24B required 179 API calls; each of the other eight models required 20.Source: §I.5 · p. 25

J Persona-based Probability Distributions

  • The persona-based probability matrix spans demographic personas and destination outcomes, showing how demographic factors affect distributional structure.The displayed Indianapolis matrix organizes 20 personas by age and education against destination outcomes.Source: J Persona-based Probability… · p. 25

K Extended Prompt Sensitivity Analysis

The extended analyses compare model choice with prompt perturbations, test seasonal and temperature effects, and assess whether the findings persist under greedy decoding. Across these tests, the supplied results show model-scale shifts exceeding persona-conditioning shifts and systematic temperature effects on several metrics.

K.1 Model Sensitivity vs Prompt Sensitivity

  • Cross-model JS divergences (0.365–0.501) exceed persona-conditioning divergence (0.134); model shifts are reported as 2–4× larger than persona shifts.Age and education perturbation divergences are 0.330 and 0.261, respectively.Source: §K.1 · pp. 25–26

K.2 Seasonality Perturbation Details

  • Across four airports, mean seasonal divergence is 0.098, below age (0.33), context (0.31), education (0.26), and persona (0.13) point estimates.The passage cautions that this is an ordering of point estimates, not a separation, because no confidence interval was estimated for the four-airport mean.Source: §K.2 · p. 26, Table 12 · p. 26

L.1 Experimental Design

  • Temperature tests covered 10 values across six airport-persona settings, using 10 iterations per combination and 600 API calls overall.The appendix reports cross-setting metrics by temperature in Table 13.Source: §L.1 · p. 26, §L.2 · p. 26

L.2 Temperature Summary Table

  • Entropy increased and top-5 mass decreased in all six settings as temperature rose; Jaccard@5 declined overall but not monotonically.The table notes that Jac@10 also shows mode-set stability across the sampled temperature range.Source: Table 13 · p. 27

L.3 Statistical Tests

  • Temperature effects were significant in all 18 tests for max probability, entropy, and top-5 mass (p < 0.001), with entropy rising and top-5 mass falling in 6/6 settings.Ranking stability declined in 5/6 settings; Jaccard@5’s aggregate decline was r = −0.930 across ten cross-setting means (p = 0.0001).Source: §L.3 · p. 27

M.1 Experimental Design

  • Greedy decoding preserves all qualitative findings across the three core claims when compared with stochastic decoding at T=0.2.Source: Table 14 · p. 27

M.1 Experimental Design

Greedy decoding retains the reported convergence, discordance, and perturbation patterns. The robustness appendix tests variation from ordering, paraphrase, and origin, and decomposes metric variance across factors.

M.1 Experimental Design

  • At T=0, modes-first convergence is faster (iteration 2 versus 3 for IND), while scale discordance persists and intensifies.The 8B model still produces 100 unique destinations; inter-model correlation is r = 0.056.Source: §M.1 · p. 27

N.1 Three Robustness Sources

N.2 Coefficient of Variation by Source

  • Table 15 compares coefficient of variation across the three robustness sources; its caption characterizes values below 21% as good reproducibility.The supplied passages do not give the individual Table 15 values.Source: §N.2 · p. 28, Table 15 · p. 28

N.4 Consensus Destinations

  • Across 15 airport-list orderings, ORD appears in the top five 15 times, followed by DFW (14/15), MCO (12/15), LAX (11/15), and DEN (10/15).Source: §N.4 · p. 28

N.3 Variance Decomposition (ICC)

  • The ICC(1,1) decomposition compares departure airport, persona, and model scale as contributors to metric variance.Model scale is the only factor with a non-trivial positive component and is largest for distributional support; departure airport stays at or below 0.08.Source: §N.3 · p. 28, Table 16 · p. 29

Related papers

Loading 2610.01000v1…