PaperlayerView exact revision on arXiv

Source-linked AI summary

3, 7, 14, 17, 42: Instruction Tuning Installs and Sharpens Number-Choice Heuristics in Language Models

Samar Ansari

arXiv:2610.36178v1cs.CL

TL;DR

A model’s arbitrary choices should spread probability across plausible options, but whether language models do so is measurable. The paper computes integer-choice distributions directly and finds that instruction tuning makes them less uniform, with structured shifts in number preferences.

  • Problem

    When no option is favored, a model should spread probability across plausible choices, making arbitrary-choice behavior a measurable calibration question.

    Source: §1 · p. 1

  • Method

    The authors sum probabilities over each integer’s scored spellings to compute distributions without sampling error across open-weight model lines and decoding conditions.

    Source: §1 · p. 2

  • Results

    All 15 usable checkpoint pairs move farther from uniform after instruction tuning, which installs round-number avoidance and sharpens endpoint avoidance.

    Source: §1 · p. 2, §9 · p. 16

  • Takeaways & Limitations

    Do not use language models as random number generators; behavior on nonnumeric arbitrary choices remains untested.

    Source: §8 · p. 15

    The evaluation uses English prompts only, limiting its scope across languages.

    Source: Limitations · p. 16

Abstract

from arXiv · show

We measure, without sampling error, the probability a language model assigns to each integer from 1 to N when asked for a random number. For a fixed prompt the model defines one probability vector, which we compute by summing over each integer's scored spellings rather than estimate by sampling, removing the noise that limits earlier studies [1, 4]. We evaluate 18 open-weight model lines at five ranges up to N = 100, under 29 decoding configurations, on two inference stacks; 15 of them have matched base and instruction-tuned checkpoints that both give usable answers. Instruction tuning makes the distribution less uniform in all 15 pairs. It installs one heuristic, round-number avoidance, and sharpens another, endpoint avoidance. Base checkpoints do not avoid round numbers once the endpoints are removed from those sets; tuned checkpoints avoid them, multiples of 10 almost completely. Base checkpoints return the endpoints 1 and N at 0.30 to 0.60 of the interior rate; tuned checkpoints almost never return them. A preference for odd numbers is consistent across tuned checkpoints only at N = 100, where base checkpoints also show it weakly. Endpoint avoidance and the odd preference fit the account Kubovy and Psotka [6] proposed for human number choice; round-number avoidance is not part of that account, and the midpoint avoidance it predicts is essentially absent, with one slight exception at N = 100. Tuning does not install a new favourite integer shared across models: above N = 10 no modal integer is shared by more than 42 percent of tuned conditions, and P (17) rises in 10 of 15 pairs and falls in 5. The concentration is present in the model's own distribution: at temperature 1.0, nucleus truncation at top-p 0.95 adds a median of 0.035 to the total variation distance.

1 Introduction

The paper treats arbitrary number choice as a calibration question and replaces sampled answer counts with exact model-assigned distributions. Across matched checkpoints, instruction tuning makes distributions less uniform and installs or sharpens distinct number-choice heuristics.

1 Introduction

  • Earlier sampling-based studies could not separate the model’s assigned distribution from decoder effects or resolve fine-grained probability differences at typical call budgets.The paper claims novelty for its measurement and decomposition, not for the finding that base models are closer to uniform than tuned models, reported previously from sampled data [8].Source: §1 · pp. 1–2
  • The measurement sums probabilities for each integer’s bare and leading-space spellings, yielding exact answer distributions without sampling error.The study covers 18 open-weight model lines, five ranges through N = 100, 29 decoding configurations, and two inference stacks.Source: §1 · pp. 1–2
  • All 15 usable base/instruct pairs became less uniform after tuning, which introduced round-number avoidance and pushed existing endpoint avoidance to near-total levels.Base models return endpoints at 0.30 to 0.60 of the interior rate; tuned models return them at 0.000 to 0.001.Source: §1 · p. 2
  • A consistent odd-number preference appears across tuned checkpoints only at N = 100, where base checkpoints also show it weakly.Endpoint avoidance and odd preference fit Kubovy and Psotka’s account; round-number avoidance is not part of it.Source: §1 · p. 2
  • The contributions include a controlled comparison of 15 checkpoint pairs and a public dataset of exact distributions covering 85 completed runs and 12,325 result rows.The enumeration method was checked against 5,000 sampled generations for one condition and replicated on a second hardware stack.Source: §1 · p. 2

2 Related Work

The related work situates single-number choice within human number-choice research and prior language-model evaluations, then identifies the need for exact distributions and controlled comparisons. The paper positions enumeration as a way to address gaps left by sampling-based studies.

2.1 Human number choice

  • Human number-choice research links spontaneous-looking answers to avoidance of endpoints, even numbers, multiples of 3, and midpoints, while the paper compares models against an exact uniform baseline.Kubovy and Psotka proposed these patterns as a conjecture; Towse et al. studied sequences rather than matched single choices over larger ranges.Source: §2.1 · p. 2

2.2 Language models as random number generators

  • Prior language-model studies found non-uniform random-number output, while Coronado-Blázquez [1] sampled six models across ranges, languages, and temperatures.The paper reports matching prior modes at N = 5 and N = 10, but only partial agreement at N = 100; West and Potts [8] reported base models closer to uniform than aligned models.Source: §2.2 · p. 3

2.3 What is missing

  • The paper identifies sampling’s limits: it does not directly measure valid-integer probability or separate the model’s distribution from decoder effects, and it lacks matched base/instruct comparisons with answer position fixed.Enumeration applies candidate-answer scoring to open-ended choice with a stop factor; at N = 30, top-p 0.95 adds a median 0.035 to total variation distance at temperature 1.0.Source: §2.3 · p. 3

3 Method

The method enumerates each integer’s probability from scored token sequences, including the probability that the model stops after the answer. It applies decoding and prompting conditions, validates the measurements, and specifies exclusions and metrics.

3.1 Enumeration

  • Enumeration sums the probabilities of scored spellings for each integer, including an immediate-stop factor so each answer is complete and self-contained.Both bare and leading-space spellings are scored; non-standard spellings are omitted, so the resulting probability is a lower bound on the model’s full probability of writing that integer.Source: §3.1 · pp. 3–4

3.2 Decoding transforms

  • Seven temperatures and four truncation settings, plus greedy decoding, yield 29 configurations by recomputing per-position probabilities from stored logits.The model prior is temperature 1.0 with no truncation.Source: §3.2 · p. 4

3.3 Prompts and conditions

  • Three prompting conditions are used, with base completions compared against instruction-tuned prefill to avoid confounding post-training with forced answer position.Qwen3 results use a non-default thinking-off condition and may not carry over to its default thinking mode.Source: §3.3 · p. 4

3.4 Validation

  • For Qwen3-0.6B at N=30 and temperature 0.7, enumeration and 5,000 samples give the same modal integer, 15, while random-number generators yield TV distances near zero.The enumerated and sampled modal probabilities are 0.579 and 0.587; the random-generator TV distances are 0.0040 and 0.0087 at 200,000 draws.Source: §3.4 · p. 5

3.5 Exclusions and metrics

  • The pre-specified valid-mass thresholds exclude rows below 10^-4 and restrict model-prior analyses to rows at or above 0.10; uniformity is measured with TV distance and entropy deficit.The first threshold excludes 1,247 of 12,325 rows (10.1 percent); cross-range comparisons also use TV divided by 1 − 1/N.Source: §3.5 · p. 5

4 Measurement Validity

The two inference stacks usually select the same modal integer, but numeric-format differences can cause concentrated failures. The comparison is limited to conditions with valid output on both stacks.

4 Measurement Validity

  • With identical weights, prompts, and decoding settings, the stack comparison tests precision sensitivity as well as reproducibility.The stacks use bfloat16 on CUDA and float16 on Metal.Source: §4 · p. 5
  • 26 of 49 conditions at N = 30 produced valid output on both stacks, limiting the comparison to that subset.Seven model lines were measured on Metal only, and three Gemma checkpoints returned no valid probabilities on Metal.Source: §4 · p. 5
  • 96.2% of 3,715 overlapping stochastic model-conditions had the same modal integer across stacks; median total variation distance was 0.0167.Mode agreement ranged from 93.2% at T = 0.1 to 97.8% at T = 0.8.Source: §4 · pp. 5–6
  • Low-temperature agreement is usually exact, but near-tied top candidates can produce near-maximum distribution disagreement.At T = 0.1, the 90th-percentile distance was 0.162, compared with 0.063 at T = 0.8.Source: §4 · p. 6
  • 117 of 125 greedy conditions agreed across stacks; when a top tie flips the argmax, the distance can reach 1.000.The median and 90th-percentile differences were both 0.Source: §4 · p. 6
  • Nucleus truncation can amplify small arithmetic differences: at T = 0.7, one condition retained 5 integers on one stack and 11 on the other.At T = 1.0, the same condition retained 21 integers on both stacks; individual truncated conditions should be reported per stack.Source: §4 · p. 6

5 Instruction Tuning Concentrates the Distribution

Across the usable paired checkpoints, instruction tuning makes answer distributions less uniform, sometimes nearly deterministic. The concentration target varies by model, and truncation effects are generally smaller than the tuning effects.

5 Instruction Tuning Concentrates the Distribution

  • Some base checkpoints were near uniform, while several tuned checkpoints were close to deterministic at N = 30.OLMo-2-7B base had a total variation distance of 0.063; gemma-3-4b-it assigned probability 1.000 to 17 with prefill.Source: §5 · pp. 6–7
  • All 15 usable base–instruction-tuned pairs moved away from uniform on both total variation distance and entropy deficit.At N = 30, gemma-3-4b moved from a total variation distance of 0.135 to 0.966.Source: §5 · p. 6, Table 2 · p. 7
  • The integer receiving the concentration is model-specific and is not predicted by the base checkpoint.P(17) rose for gemma-3-4b but fell for Qwen3-1.7B, whose tuned model instead concentrated on 15.Source: §5 · p. 7
  • At temperature 1.0, top-p = 0.95 added a median total variation distance of 0.035, small relative to the tuning effects.The concentration was already present in the model’s distribution before truncation.Source: §5 · p. 7
  • At temperature 0.7, Qwen3-0.6B in chat mode had stack-dependent truncation effects, so single-condition sampler effects are specific to stack and temperature.Its support fell from 30 integers to 5 on CUDA and 11 on Metal; pooled medians, rather than single conditions, are reported as model properties.Source: §5 · p. 7

6 Which Integers Survive

The section compares integer-set probability mass with the uniform share, where 1.00 means parity and lower ratios indicate avoidance. Instruction tuning strengthens endpoint avoidance, introduces round-number avoidance, and leaves only a qualified odd preference; midpoint avoidance is largely absent.

6.1 Endpoint avoidance

  • Tuned checkpoints virtually eliminate both endpoints, while base checkpoints return them at roughly 0.30–0.60 of the interior rate.The endpoint ratios use the average interior probability as the reference; the avoidance is roughly symmetric for 1 and N.Source: §6.1 · pp. 8–9

6.2 Round-number avoidance

  • Base checkpoints show no endpoint-corrected round-number avoidance, while tuned checkpoints avoid both classes, especially multiples of 10.At N = 30, multiples of 10 fall from 1.12 to 0.06 of uniform share after tuning, versus 1.33 to 0.59 for multiples of 5 that are not multiples of 10.Source: §6.2 · pp. 9–10

6.3 Odd-number preference, and why controls were needed

  • After controls, the tuned odd preference is consistent across models only at N = 100, while base models also show a smaller preference there.The controls remove multiples of 5, endpoints, and the modal integer; removing the top two or three integers leaves tuned medians above parity but does not make the preference uniform across models.Source: §6.3 · p. 11

6.4 Midpoints are essentially not avoided

  • Midpoints are essentially not avoided; only tuned models at N = 100 place the midpoint slightly below the middle of its comparison set.The comparison uses the middle integer that is not a multiple of 5, since the midpoint itself is often a multiple of 5.Source: §6.4 · pp. 12–13

6.5 What we cannot resolve

  • The prime contrast remains unresolved because its non-prime comparison set largely overlaps multiples of 3, and the ranges stop at N = 100.Separating the prime and multiple-of-3 hypotheses would require a range such as N = 1000.Source: §6.5 · p. 13

7 Range Dependence and the Modal Integer

The modal integer varies with range rather than tracking a fixed fraction of it, and tuned models show limited agreement on a shared mode above N = 10. Concentration remains substantial across ranges, while tuning has no consistent effect on the probability of 17.

7 Range Dependence and the Modal Integer

  • Only 36%, 42%, and 39% of tuned conditions share the leading mode at N = 20, 30, and 100, respectively.The corresponding leading modes are 14, 17, and 42; their ratios to the range maximum vary from 0.42 to 0.70 across the five ranges.Source: Table 6 · p. 13, §7 · p. 13
  • At N = 100, the earlier sampled modes 37, 47, and 73 appear among tuned conditions but none leads; 42 leads in 13 of 33.At N = 5 and N = 10, the exact distributions closely reproduce the earlier reported modes.Source: §7 · p. 13
  • At N = 100, median total variation distance is 0.842 for instruct checkpoints and 0.241 for base checkpoints.After dividing by the maximum attainable distance, neither series rises steadily with range; the gap remains between 0.53 and 0.63.Source: §7 · pp. 13–14
  • Tuning raises P(17) in 10 of 15 pairs and lowers it in 5, with no consistent 17 effect.The median ratio to uniform is 1.71 for base and 2.22 for instruct checkpoints, but the pairwise direction is mixed.Source: §7 · pp. 14–15, Table 7 · p. 15

8 Discussion

The discussion finds only partial overlap between tuned-model number choices and the proposed human strategy, alongside a distinct round-number avoidance pattern. It cautions against using models as random generators and describes external sampling or renormalization as remedies.

8 Discussion

  • Tuned models share endpoint avoidance and a controlled odd preference only at N = 100 with the human account, but essentially do not avoid midpoints.Round-number avoidance is the clearest new behavior and is absent from base checkpoints; the proposed role of human habits in post-training data was not tested.Source: §8 · p. 15
  • Do not use a language model as a random-number generator: tuned models systematically avoid endpoints and round values when choosing among integers.Whether this applies to arbitrary choices among nonnumeric options is untested and remains a conjecture.Source: §8 · p. 15
  • When distributions are inspectable, renormalize logits over valid candidates before sampling; otherwise, use an external generator.The paper presents this as a one-line remedy, not a research contribution.Source: §8 · p. 15
  • Sampling validation covered only one model in one condition, and cross-stack agreement would not catch an enumeration error shared by both stacks.The authors recommend sampling validation for at least one condition per model as a companion to enumeration.Source: §8 · p. 15

Limitations

The study’s scope and interpretation are bounded by its model coverage, prompt conditions, and measurement choices. Several comparisons also have limited validation or small and overlapping integer sets.

Limitations

  • The study covers open-weight models no larger than 8B parameters; closed models, whose behavior is practically important, are not measured.Model-size effects are not reported because parameter count is confounded with tokenizer, corpus, architecture, and post-training recipe.Source: Limitations · pp. 15–16
  • Sampling validation covers one model in one condition, and non-standard integer spellings are excluded from the measured probability mass.Scored spellings are the bare and leading-space forms; omitted spellings were not measured.Source: Limitations · p. 16
  • Only one prompt is primary, prompts are English, and base and instruct formats cannot be made identical despite controlling answer position.Prompt variants were not run at scale; prior work found language effects [1].Source: Limitations · p. 16
  • Statistics are descriptive, cross-model sign tests assume independence, and quantisation was not varied beyond bfloat16 or float16.Three Gemma 2 9B runs were incomplete; some model lines ran on only one stack, and Qwen3 was measured with its reasoning block closed.Source: Limitations · p. 16
  • Round-number sets contain only one to three members at N ≤ 30, so only the N = 100 column rests on many integers.The prime hypothesis is also confounded with multiples of 3 and remains unresolved.Source: Limitations · p. 16

9 Conclusion

Instruction tuning shifts all 15 usable base–instruct pairs away from uniformity, installing round-number avoidance and strengthening endpoint avoidance. Other preferences are limited or model-specific, and the observed concentration is present in the model’s own distribution.

9 Conclusion

  • All 15 usable pairs move farther from uniformity after instruction tuning, while the target of concentration remains model-specific rather than predictable from the base checkpoint.At temperature 1.0, nucleus truncation adds a median of 0.035 and at most 0.082 to distance from uniform; the large sampler effect occurs at temperature 0.7 and differs by stack.Source: §9 · p. 16
  • Tuned models avoid round numbers while base models do not; endpoints already occur at 0.30 to 0.60 of the interior rate in base checkpoints and almost never after tuning.Round-number avoidance is not part of the human-choice account proposed by Kubovy and Psotka.Source: §9 · p. 16
  • A tuned-model preference for odd values is shown by most models only at N = 100, where base checkpoints also show it weakly; midpoint avoidance is essentially absent.Endpoint avoidance and the odd preference fit Kubovy and Psotka’s account, which predicts midpoint avoidance; one slight midpoint exception appears for tuned models at N = 100.Source: §9 · p. 16
  • Across all 15 usable pairs, instruction tuning moves integer-generation distributions away from uniform, installs one heuristic and sharpens another; specific integers chosen do not match human data beyond 7 at N = 10.Source: §9 · p. 16

Reproducibility

The authors release the distributions and analysis materials needed to reproduce the paper’s tables and figures.

Reproducibility

  • The release includes exact probability distributions for all 85 completed runs, measurement code, the runner, and analysis scripts.The authors state these materials are sufficient to reproduce every table and figure.Source: Reproducibility · p. 16
  • The released materials are stated to reproduce every table and figure in the paper.Source: Reproducibility · p. 16
  • The repository is available at https://zenodo.org/records/23674051.Source: Reproducibility · p. 16

A Excluded Rows

The appendix accounts for every row removed by two exclusion thresholds fixed before analysis. The primary threshold excludes rows whose valid mass is too small for a meaningful conditional distribution.

A Excluded Rows

  • Both exclusion thresholds were fixed before analysis, and the appendix accounts for every removed row individually or by group.Source: A Excluded Rows · p. 16

A.1 The primary threshold

  • The primary threshold excludes rows with valid mass below 10^-4 because their conditional distributions are numerically meaningless.It removes 1,247 of 12,325 rows (10.1 percent), leaving 11,078.Source: §A.1 · p. 17
  • The excluded rows include greedy outputs that begin with preamble words, reasoning-model runs with the reasoning block open, Mistral prefill runs that rarely stop after an integer, and specified Metal-stack gemma runs.For reasoning-model runs the valid mass is around 6 × 10^-15; Mistral prefill rows have valid mass below 5 × 10^-5. The affected gemma checkpoints are valid on CUDA, whose rows are used.Source: §A.1 · p. 17

A.2 The set-membership threshold

For model-prior analyses, a stricter valid-mass threshold excludes rows that describe only rare instruction-following outputs. The appendix also identifies incomplete runs and the resulting model coverage.

A.2 The set-membership threshold

  • A valid mass of at least 0.10 is required for model-prior analyses because low-mass conditional distributions describe only rare occasions when the model follows the instruction.The threshold avoids comparing such rows with rows whose valid mass is near 1.000.Source: §A.2 · p. 17
  • At N = 30, the threshold removes three model-conditions, listed in Table 8; Mistral-7B-v0.3 base would otherwise appear most uniform at total variation distance 0.061, despite valid mass of 0.027.After excluding it, the most uniform retained condition is OLMo-2-7B base at 0.063 with valid mass 0.995.Source: §A.2 · p. 17, Table 8 · p. 17
  • The same three conditions fail the threshold at every range, leaving 16 base and 33 instruct conditions per range and 15 pairs for paired and P(17) comparisons.The excluded gemma-2-2b pair moves in the same direction as the others, from total variation distance 0.106 to 0.542.Source: §A.2 · p. 17

A.3 Runs not completed

  • Three Gemma 2 9B runs were not completed because storage ran out, so the 9B size point is missing although the model line is represented by 2B checkpoints.Source: §A.3 · p. 17

B Models and Stacks

  • The attempted set comprised 18 model lines, including Phi-4mini-instruct without a public base checkpoint; 15 of 17 paired lines enter the paired comparison.Mistral-7B is lost to the primary threshold and gemma-2-2b to the set-membership threshold. The two stacks completed 53 and 32 runs, respectively, for 85 runs and 12,325 rows.Source: B Models and Stacks · p. 18, Table 9 · p. 18

Related papers

Loading 2610.36178v1…