PaperlayerView exact revision on arXiv

Source-linked AI summary

Ask a Language Model for Lottery Numbers: Concentration in Repeated Six-of-49 Outputs

Dmitrij Żatuchin

arXiv:2610.00052v1cs.IRcs.AIcs.CL

TL;DR

Because run-to-run variation can be mistaken for randomness, the paper evaluates language-model lottery outputs against uniform six-of-49 sampling and archived Lotto draws. Across tested configurations, modal tickets accounted for 22.5%–68.0% of valid responses, a concentration incompatible with independent uniform selection.

  • Problem

    Because sampling temperature makes model outputs vary from run to run, the study asks whether that variation reflects independent uniform sampling of six distinct numbers.

    Source: §1 · p. 1

  • Method

    The study compares model-output number frequencies with uniform six-of-49 simulations and archived Polish Lotto samples, using effective diversity and a Monte Carlo test.

    Source: §1 · p. 1

  • Results

    Modal unordered tickets accounted for 22.5%–68.0% of valid responses across the six tested configurations, incompatible with independent uniform selection.

    Source: §5 · p. 4

  • Takeaways & Limitations

    Run-to-run variation alone is no evidence that model outputs cover the intended answer space.

    Source: §5 · p. 4

    The experiment does not identify the cause of concentration, and repeated calls were not verified to bypass provider-side response caches.

    Source: §4 · p. 4

Abstract

from arXiv · show

We evaluate six language-model configurations on requests for six distinct random integers from 1-49. Across 1,200 attempted calls using four English prompt variants, 1,184 responses yielded valid tickets. Effective diversity of number frequencies ranged from 9.9 to 18.0, compared with simulated fifth-percentile thresholds of 46.6-46.7 under independent uniform six-of-49 sampling at the corresponding sample sizes. Systems produced 8-93 distinct unordered tickets, and their modal tickets accounted for 22.5-68.0% of valid responses. Two archived Polish Lotto samples provided a physical-lottery comparison, with effective diversities of 41.1 and 41.4 at smaller sample sizes. These results demonstrate substantial concentration under the tested deployment settings. They do not identify its mechanism or establish performance under other prompts, temperatures, or tool configurations.

1 Design and benchmarks

The study compares model-generated six-number tickets with independent uniform sampling and archived Polish Lotto draws, using effective diversity and Monte Carlo testing to assess concentration. It tests provider-default configurations across four English prompts and presents per-number distributions and ticket-level behavior.

1 Design and benchmarks

  • Six engines received 200 attempted calls each across four English prompts, with fresh contexts and provider-default settings; Table 2 covers valid answers pooled across prompts.The experiment did not systematically vary decoding settings; sonar used retrieval by default, unlike the other five configurations.Source: §1 · p. 1, Table 2 · p. 3
  • Effective diversity is the reciprocal concentration of number frequencies: equal use of all 49 numbers scores 49, while always returning the same six scores 6.Finite uniform samples fall below 49; simulated medians are 38.5 at 26 tickets and 41.5 at 39.Source: §1 · p. 1
  • 20,000 simulated uniform-ticket datasets per source establish Neff floors and one-sided Monte Carlo p-values; the top-number count remains descriptive because an OR test is too liberal.The combined rule’s false-positive rate is 6.1–7.7%, above the nominal 5%.Source: §1 · p. 1, Table 1 · p. 3
  • Figure 1 uses a shared scale for number shares, marking modal-ticket numbers and the 2.04% uniform share across archival and engine panels.Engine panel order is presentational, not a quality ranking.Source: Figure 1 · p. 2

2 Results

Across the tested engines, number frequencies and whole-ticket outputs were concentrated, while archived Lotto samples were compatible with the sample-size-specific uniform benchmark. Results also varied across prompts, and ordering and number-range summaries describe separate output characteristics.

Spread of numbers

  • Neff was 9.9–18.0 for the six engines versus simulated fifth-percentile floors of 46.6–46.7, while archived samples scored 41.1 and 41.4 and were benchmark-compatible.Each engine had pMC = 0.00005; the archived samples had pMC = 0.95 and 0.49.Source: §2 · p. 3

Tickets

  • gpt-5.6-luna produced just 8 distinct tickets in 200 valid answers, with its modal ticket appearing 101 times; sonar’s modal share was 68.0%.The probability of any repeated ticket in 200 independent uniform draws is 0.0014.Source: §2 · p. 3

Prompt variants

  • Concentration appeared in all 24 prompt-variant cells, but gpt-5.6-luna had 2–3 distinct tickets and a 96% modal share under base and lottery phrasings, while grok-4.5 shifted from 8% to 72% modal share when “lottery” appeared.The pooled results describe the specified mixture of four prompts.Source: §2 · p. 3

Ordering

  • Sorted answers ranged from 27.7% for mistral-large to 100% for gemini-3.7-flash and gpt-5.6-luna; ticket identity is analyzed as an unordered set.claude-sonnet-5 sorted 99.5%, sonar 99.0%, and grok-4.5 87.0% of parsed answers.Source: §2 · p. 3

Number 42

  • The number 42 ranged from fifth of 49 for gemini-3.7-flash, grok-4.5, and mistral-large to absent from gpt-5.6-luna’s 1,200 numbers; its archived ranks do not support a pattern claim.In the archived samples, 42 was drawn once each and ranked 48th and 49th, consistent with chance at these sample sizes.Source: §2 · pp. 3–4

Number ranges

  • Numbers 1–31 made up 63.8%–68.0% of model outputs versus a uniform share of 63.3%; archived samples were 67.3% and 68.8%.Mean returned numbers per engine ranged from 24.42 to 27.00 against a uniform expectation of 25.0.Source: §2 · p. 4

Data quality

  • Of 1,200 attempted calls, 1,184 yielded valid tickets after 12 rate-limit failures and four unparsable claude-sonnet-5 answers; no parsed answer had duplicate or out-of-range numbers.The 12 HTTP 429 failures occurred on mistral-large during the parallel run and were a collection artifact.Source: §2 · p. 4

3 Related work

Prior work documents variability in repeated model answers, biased number generation, and audits of repeated responses. This study adds six-of-49 tickets, their whole-ticket concentration, and a physical-lottery comparison at matched small sample sizes.

3 Related work

  • Earlier work documents differing answers to identical prompts [1], biased coin-flip and number-range outputs [2] [3], and protocols for auditing repeated queries.Another line decomposes variance in repeated brand answers [4].Source: §3 · p. 4
  • This study adds the six-of-49 task and analysis of concentration in complete unordered tickets.It also compares model outputs with physical-lottery samples at matched small sample sizes.Source: §3 · p. 4
  • The identical modal ticket in two systems is reported as a descriptive observation.The passage does not attribute a cause to this match.Source: §3 · p. 4

4 Scope and limitations

The study rejects independent uniform sampling only for the configurations tested and does not identify why their outputs concentrate. The physical-lottery archive supports marginal number-count comparisons, not claims about repeated whole tickets.

4 Scope and limitations

  • The tested outputs reject independent uniform six-of-49 sampling, but the study does not distinguish among memorisation, decoding, prompt conditioning, retrieval, routing, or caching as causes.Provider-side cache bypass was not verified; a cache returning stored answers for repeated prompts would concentrate outputs by construction.Source: §4 · p. 4
  • The six systems used two tool configurations: sonar had retrieval enabled by default, while the other five received no search parameters.The passage cautions against inferring one shared mechanism across the systems.Source: §4 · p. 4
  • The archived physical-lottery data contain per-number counts but no draw-level ticket composition, so repeated whole-ticket claims apply only to the engines.The samples cover matched June–August windows in 1999 and 2026, 27 years apart.Source: §4 · p. 4

5 Conclusion

Every tested configuration placed 22.5–68.0% of valid responses on one modal unordered ticket, while the two archived Lotto samples were compatible with their sample-size-specific benchmarks. Proposed next steps include testing other languages and decoding settings, commercial questions, and ticket structure.

5 Conclusion

  • 22.5–68.0% of valid responses across the six tested configurations fell on the modal unordered ticket, incompatible with independent uniform six-of-49 selection.The two archived Lotto samples were compatible with the simulation benchmarks used here.Source: §5 · p. 4
  • The Polish collection arm is built but unrun, and future work proposes testing other languages because favourite numbers may be locale-specific.Temperature sweeps and fixed-seed sampling are also proposed to examine how effective diversity and ticket-set size change.Source: §5 · p. 4
  • A future application would test the design on commercial questions, where the concentrated set is brands; a companion note measures that repertoire..Source: §5 · p. 4
  • The near-even gaps between numbers in modal tickets invite formal description, and ticket-level comparison with human favourite-number literature remains a proposed direction.Source: §5 · p. 5

Declarations

The author discloses a company interest. Raw answers, counts, metrics, collector, and recomputation code are public, and Lotto baselines derive from operator statistics archived as count vectors.

Declarations

  • The author is CEO of Rankfor.AI, which sells AI-visibility measurement; the note is methodological and names no customer.Source: Declarations · p. 5
  • Raw answers, per-number counts, metrics, the collector, and the recomputation script are public.Source: Declarations · p. 5
  • The Lotto baseline uses operator statistics for June–August 1999 and 2026, archived as per-number count vectors.The archive permits recomputation from integer counts.Source: Declarations · p. 5

A Prompt variants and per-variant results

Four English prompt variants each received 50 attempted calls per cell; Table 3 reports effective number-frequency diversity and modal share for each variant.

B Deployment record

The deployment record documents when and how the per-engine calls ran, which settings were sent, and the returned model identifiers.

  • All calls ran on 1 September 2026, 09:05 to 09:08 UTC, with one user message and a fresh context per call.Source: Table 4 · p. 6
  • No temperature, seed, top-p, or search parameters were sent; decoding defaults are undisclosed, while sonar retrieves by default.Source: Table 4 · p. 6
  • Each engine’s API response returned the same model ID across all 200 calls.Source: Table 4 · p. 6
Loading 2610.00052v1…