PaperlayerView exact revision on arXiv

Source-linked AI summary

Shared and structured inputs undermine collective random choice by reasoning AI agents

Takahiro Ezaki, Naoto Imura, Katsuhiro Nishinari

arXiv:2610.09667v1cs.AIphysics.soc-ph

TL;DR

Shared or structured selection inputs can increase collective loss even when average rates remain plausible, and choices derived from similar information create vulnerability. For oversight, the paper connects reliable random selection with controlling the source and sharing of inputs and evaluating rates, dependence, and predictability.

  • Problem

    For oversight, predictability matters because a sender who controls selection inputs could choose requests of concern even under policies with plausible rates.

    Source: Discussion · p. 6

  • Method

    The paper considers random selection for oversight and draws on supplied-draw evidence to discuss trusted-code randomness and fresh draws for each decision.

    Source: Discussion · pp. 6–7

  • Results

    Shared or structured inputs can increase group loss despite plausible average rates, creating collective vulnerability when agents use the same or similarly structured information.

    Source: Discussion · pp. 6–7

  • Takeaways & Limitations

    Reliable random-selection policies require control over the source and sharing of selection inputs, alongside evaluation of rates, dependence, and predictability.

    Source: Discussion · p. 7

    The evidence is behavioural and identifies regularities and transfer limits; the uniform-draw test was small, leaving reliable rare-event selection in deployment unestablished.

    Source: Discussion · p. 7

Abstract

from arXiv · show

Random selection is widely used in resource allocation and auditing, making reliable implementation essential for AI-agent systems. Behavioural tests across six reasoning models uncovered threshold and divisibility rules used in identifier-based choices. For threshold-following GPT-6 Sol and Gemini 3.8 Flash, single-agent measurements prospectively predicted correlated participation under shared identifiers and biased participation under distinct identifiers with common timestamp bits. Changing dates, formats and identifier labels revealed when these predictions held. Explicit instructions to randomize independently reduced but did not eliminate shared-input correlation. To test implications for oversight, we asked four models to select customer requests randomly for human review. GPT-6 Sol approached the target rate while selecting predictably from identifiers; the others rarely selected requests. All four closely followed supplied random draws. These findings expose collective and audit vulnerabilities that selection rates alone miss, making input-dependent bias, correlation and predictability central targets for agent evaluation.

Results

The results examine how identifiers, prompt instructions, and task-irrelevant text affect models’ participation, agreement, and choices, including tests on fresh requests and supplied random draws. They also assess how selection rates relate to predictable choices and failure to meet requested rates.

Single-agent tests reveal identifier-based decision rules

  • All seven reasoning settings depended on identifiers, with threshold effects of 0.62 to 0.80 for Sol and Gemini Flash, while other models used residues or mixed rules.Threshold settings’ repeated answers agreed in 78% to 92% of cases versus about 50% for independent draws; Sol and Gemini models agreed across models in 91% to 95% on designed identifiers, versus 53% expected independently. Mixed-model groups were not tested.Source: Single-agent tests reveal… · p. 3

Input structure predicts collective correlation and bias

  • Shared identifiers drove 91% of Sol groups and 88% of Gemini groups to unanimity, while UUIDv7s produced 0.95 and 0.93 participation rates and losses 6.6 to 6.9 times the independent value.Shared-identifier losses were dominated by covariance, whereas UUIDv7 losses were dominated by mean-rate bias; without identifiers, Sol’s unanimity was 39%, near the 33% independent expectation.Source: Input structure predicts collective… · pp. 3–4

Changing dates tests whether the timestamp drives rare actions

  • With raw UUIDv7s, Sol matched the threshold prediction in all 12 date–target cells, participating in 95% of below-target and 0.5% of above-target cases.Hyphens preserved the staircase but reduced participation to 38% and 0%; Gemini followed the direction more weakly than predicted, at 41% and 0% versus 90% and 13%.Source: Changing dates tests whether the… · p. 4

An explicit audit policy exposes predictable or insufficient selection

  • Sol’s UUIDv7 selections changed with identifier format: it selected 37 of 48 raw-digit requests at 1%, none at 0.5%, and 46% to 67% when trailing bits met the designed threshold.With hyphenated UUIDv7s it selected none, while other models rarely selected time-ordered requests.Source: An explicit audit policy exposes… · p. 5

Prompt changes distinguish identifier meaning from instructions to randomize

  • Study 6 tested prompt features separately and in combination, with concurrent controls, to determine whether wording invited input use or identifier meaning mattered.Source: Prompt changes distinguish identifier… · p. 5
  • Shared agent_id raised loss by 0.17 for both models; under record_id, the rise shrank by 0.078 for Sol and 0.169 for Gemini.Unchanged study 2 prompts reproduced prior answers in 91% to 96% of cases, versus 54% to 57% expected for independent draws.Source: Prompt changes distinguish identifier… · p. 5
  • Shared identifiers raised loss by 0.12 and 0.06 (0.17 without the sentence), with unanimity of 61% and 69% versus 18% and 40% expected (Fig. 5a,b).With private identifiers, threshold effects were 0.67 for Sol (0.92 without the sentence) and 0.38 for Gemini.Source: Prompt changes distinguish identifier… · p. 5
  • Sol’s UUIDv7 participation was 0.93 and 0.96 with agent_id versus 0.54 and 0.16 with record_id; when both fields were supplied, agents favoured their own identifier, with effects 0.76 and 0.72.Source: Prompt changes distinguish identifier… · p. 5

Fresh requests test rule stability and external randomization

  • Sol selected 41 of 48 requests with identifiers below 1% of their range, 3 of 48 with low last 16 bits, and none with neither; the last-16-bit rule failed its pre-specified criterion.The threshold rule held on fresh requests.Source: Fresh requests test rule stability and… · p. 5
  • Among 1,008 ordinary-identifier requests, Sol selected 17 (1.7%): 17 of 18 below the 1% threshold and none of the other 990.Source: Fresh requests test rule stability and… · p. 5
  • All four models followed unstratified uniform random draws in 1,004 to 1,008 of 1,008 answers, while all selected the six draws below 1%.Gemini and Claude also selected four and one requests, respectively, whose draws were above the threshold.Source: Fresh requests test rule stability and… · p. 6

Text-input tests extend the evidence beyond identifiers

  • Without identifiers, Sol’s choices depended on task-irrelevant Japanese subject lines; reasoning lowered loss more with different than shared lines, with an interaction of 0.029 (98.3% CI 0.016 to 0.043).Prospective follow-ups associated post-collection features with choices, but results had limits across languages and prompt wordings.Source: Text-input tests extend the evidence… · p. 6

Discussion

The discussion links individual decision rules to collective outcomes: shared inputs can increase group loss despite plausible average rates, while structured inputs can also shift those rates. It argues that reliable random selection requires controlling selection inputs and evaluating calibration, dependence, and predictability separately.

Discussion

  • Linking single-agent profiles to prospective group predictions reveals a route to input dependence beyond identifier-based choices.An agent’s use of visible identifier bits need not implement the requested policy.Source: Discussion · p. 6
  • Shared inputs can increase group loss while average rates remain plausible; structure in distinct inputs can instead shift the average away from target.These are different failures and call for different checks.Source: Discussion · p. 6
  • Correcting average rates alone cannot establish independence, so agent evaluation should assess calibration, dependence, and predictability separately.Repeating inputs within and across models and comparing agreement with independent-choice expectations can test whether choices remain input-dependent.Source: Discussion · p. 6
  • Evaluation should use the intended task and prompt, including shared record fields, and check whether models agree on the same inputs.Field labels, formats, and policy wording belong in these evaluations; mixed-model groups remain untested.Source: Discussion · p. 6
  • Potential implications for senders controlling selection inputs are conditional, not demonstrated attacks or defences.The discussion notes that such a sender could choose requests that avoid review even when the overall audit rate looks correct.Source: Discussion · pp. 6–7
  • For independent, unpredictable selection, the discussion supports generating randomness in trusted code and giving models fresh draws for each decision.Reliable rare-event selection in deployment remains unestablished because the uniform-draw test was small.Source: Discussion · p. 7
  • The evidence is behavioral and does not identify internal algorithms; the studies used closed models and do not establish effects in deployed systems.The reported versions, single-run studies, and synthetic inputs without tools or history limit the scope of the findings.Source: Discussion · p. 7

Methods

The methods profile identifier-based choice rules across model settings, then test whether those profiles can predict group participation. A separate audit task examines random selection for human review.

Models and access

  • Six models from three developers were accessed through public APIs, with reasoning settings that the paper notes are not comparable across developers.Each request was sent once, and most responses were constrained to one offered choice by JSON schema.Source: Models and access · pp. 7–8

Group participation task

  • Agents chose whether to participate in a group seeking to minimize the squared gap between its participation fraction and target rate.The independent-choice reference assumes a common participation probability; agents were not told the group size.Source: Group participation task · p. 8

Identifiers

  • Identifier analyses used a threshold feature for values below the target and a residue feature for identifiers divisible by the reciprocal of that target.The study also considered timestamp-bearing UUIDv7s and ULIDs, which encode time in leading bits.Source: Identifiers · p. 9

Study 1: single-agent profiles

  • Study 1 measured participation across threshold-by-residue cells and classified settings by effect size; identifier-preserving permutations supported effects with p ≤ 0.0002 when p ≤ 0.001.The profiles used 64 identifiers and four targets, with repeated answers at q = 1/3 for settings with reasoning.Source: Study 1: single-agent profiles · p. 9

Studies 2 and 3: groups and creation dates

Studies 2 and 3 tested whether single-agent identifier profiles predicted group participation and how identifier dates affected those profiles. Studies 4 and 5 examined identifier-based customer-review selection and supplied random draws.

Studies 2 and 3: groups and creation dates

  • Study 2 tested six conditions with groups of four at q = 1/3, comparing prospective input-feature predictions from study 1 with a marginal model.The primary comparison used the mean log predictive probability of observed group participation counts across UUID conditions.Source: Studies 2 and 3: groups and creation… · p. 10
  • Study 3 tested single-agent responses to UUIDv7s with three fixed creation dates and varying random bits across four targets and two formats.The identifiers’ value fractions formed a staircase, and predictions used study 1’s pooled threshold rates.Source: Studies 2 and 3: groups and creation… · p. 10

Studies 4 and 5: audit task

  • The audit presented requests with an identifier or number and a one-sentence customer summary without digits, asked whether to select each for human review, and used targets of 1% and 0.5%.Source: Studies 4 and 5: audit task · p. 11
  • 14 of 15 review selections across models in study 5’s natural-identifier part had identifiers among the 5.6% satisfying one of four simple rules.The rules were chosen after data collection and the result was exploratory.Source: Studies 4 and 5: audit task · p. 11

Study 6: prompt features

Study 6 varied prompt wording and identifier-field labels, while study 7 tested boundaries around identifiers, draws and independent-randomness instructions. Appendix analyses also report limits on predictions and reasoning-summary coding.

Study 6: prompt features

  • Study 6 varied group-task wording and field labels, and tested audit prompts with altered policy wording, no identifier, or an escalation option.The group task included repeated prompts and re-testing shared- and private-identifier conditions.Source: Study 6: prompt features · pp. 11–12

Study 7: boundaries

  • Study 7 tested group choices across identifier classes and labels, including an instruction to choose independently at probability 1/3.Its audit part used new ordinary UUIDv4s and supplied random draws across four models.Source: Study 7: boundaries · p. 12

Supplementary text-input tests

Pre-specification and reporting

  • Study designs and analysis plans were specified before collection but not externally preregistered; some later studies followed prior results or reviews.Intervals outside stage 1 were 95% and not adjusted for multiplicity.Source: Pre-specification and reporting · pp. 12–13

Reasoning summaries

  • Reasoning summaries mentioned parity in 80% of 381 summaries at q = 1/2 and 12% of 763 at q = 1/3; coding was exploratory and not validated by human raters.Other coded mentions included remainders in Sol’s counter-identifier summaries and shared inputs after prompt changes.Source: Reasoning summaries · p. 13

Use of AI tools

  • Claude and Codex assisted with manuscript editing and coding under the authors’ direction; authors retained scientific decisions and responsibility.Source: Use of AI tools · p. 13

Data availability

The data underlying the reported analyses are available in a minimal reproducibility package on Zenodo.

Data availability

  • The Zenodo package contains saved trial records, including prompts, experimental settings, visible model responses and parsed decisions.Internal protocols, operational logs, credentials, provider identifiers and encrypted reasoning or thought-signature fields are excluded.Source: Data availability · p. 13

Code availability

Offline analysis and figure-generation code is available in a Zenodo record, and reproduction uses saved records without model requests or API credentials.

Code availability

Supplementary Note 1: Prompts

The supplementary note documents prompts for the group-participation and audit tasks, including their output constraints, identifier fields, and wording variants.

1.1 Group participation task, English (studies 1 to 3; English arms of the text runs)

  • Agents chose between two options to minimize expected squared deviation of the group participation fraction from a target, without observing or communicating with other agents.Source: §1.1 · p. 30
  • Responses used a single-field JSON schema, with option labels and order varied and identifiers omitted or supplied in several formats.The supplied fields included raw and hyphenated identifiers, a counter identifier, and a private random number.Source: §1.1 · p. 30

1.2 Group participation task, Japanese (stage 1 and the text runs)

  • Japanese prompts included the stage 1 wording and paraphrased and restructured text-run 3 versions; related Chinese, Korean, and German prompts translated the Japanese wording.The restructured version omitted the closing sentence about available information, while the formula and JSON instruction remained unchanged in the translations.Source: §1.2 · p. 30, Figure 1 · p. 31

1.3 Audit task (studies 4 and 5)

  • Audit-task agents chose whether to route each customer request to human review or approve it automatically, under a policy specifying an approximate review rate.The prompt included a prohibition against using request identifiers or other request fields for selection in the prohibition condition.Source: §1.3 · p. 32
  • The audit prompts varied identifier types, including UUIDv4, UUIDv7, ULID, and request numbers, and included one-sentence request summaries.Source: §1.3 · p. 32
  • Study 5 added one of 21 channel-or-style sentences to natural-identifier request summaries, while the audit experiments did not perform actual human review.Source: §1.3 · p. 32

Supplementary Note 2: Cross-model screen

An exploratory screen tested reasoning models on the English group task and found identifier-based choice rules, alongside nonparticipation in some no-identifier conditions.

Supplementary Note 2: Cross-model screen

Supplementary Note 3: Sensitivity analyses

Sensitivity analyses tested the robustness of inference and decomposed group loss into bias and dependence components, while noting limits of the exploratory estimates.

Sensitivity analyses (exploratory)

  • Permutation tests gave p ≤0.0002 for whole-identifier effects, and threshold effects had p = 0.0001 across Sol, Luna, and the three Gemini settings.Residue effects had p = 0.0001 for GPT-5.4 mini and Claude and p = 0.0002 for Luna; with first answers only, the largest value was 0.0003.Source: Supplementary Note 3: Sensitivity… · p. 33
  • Study 6 paired resampling estimated shared-minus-private loss differences of 0.043 [0.011, 0.076] for Sol and 0.015 [−0.011, 0.041] for Gemini.Sol’s excess on the same 32 groups was 0.24 [0.07, 0.40] of its study 2 excess.Source: Supplementary Note 3: Sensitivity… · p. 33
  • Study 5 resampling estimated Sol’s human-review selection rate at 0.89% [0.40%, 1.59%], but intervals from 0 to 9 selected requests were considered unreliable.Resampling summaries and channel sentences separately gave [0.10%, 2.18%].Source: Supplementary Note 3: Sensitivity… · p. 33
  • Study 4’s joint permutation test gave 5.5×10−9, compared with 2.4×10−9 for the product over strata.Source: Supplementary Note 3: Sensitivity… · p. 33
  • For Fig. 4f, resampling-based intervals differed from exact intervals by at most 0.08 across the 24 requests selected for review.For 4 of 24 selected requests, the resampled interval was [0.04, 0.29] versus the exact interval [0.05, 0.37].Source: Supplementary Note 3: Sensitivity… · p. 33

Exploratory decomposition of group loss

  • 97.2% of Sol’s and 95.4% of Gemini’s total loss with time-ordered UUIDv7s came from squared bias, whereas shared-identifier loss included larger covariance components than squared bias.For shared identifiers, covariance components were 0.153 for Sol and 0.138 for Gemini, versus squared bias of 0.0005 and 0.0002.Source: Supplementary Note 3: Sensitivity… · p. 34

ployed systems

In study 5, GPT-6 Sol’s request selections depended on identifier rules, creating a potential audit vulnerability when senders can choose identifiers. A modeled 300-request audit illustrates how the low selection rate could reduce review coverage.

Identifier-based selection

  • GPT-6 Sol selected none of 989 requests meeting neither rule; study 7 confirmed the pre-specified value-fraction rule, but not the post-hoc low-16-bit rule.The rules were a value fraction below 1% and, in a later-discovered rule, last 16 bits below 1% of their range.Source: ployed systems · p. 34

Audit implications

  • At GPT-6 Sol’s 0.37% upper-bound rate for requests meeting neither rule, the modeled chance of any selection among 300 requests is 67%, versus 95% at a 1% audit.The illustration assumes independent selection at a constant rate.Source: ployed systems · pp. 34, 36
  • Repeated identifiers receiving the same decision is not itself a fault; the risk arises when senders can choose among identifiers.The study did not test an adaptive sender.Source: ployed systems · p. 36

Supplementary Note 5: Text-input studies without identifiers

The text-input studies found that shared subject lines affected group loss and that subject-line features predicted participation in some tested conditions. Prospective runs supported some Japanese-language predictions, while other predictions and generalization remained limited.

5.1 Pre-specified test of the group-level effect

  • J = 0.029 (98.3% CI 0.016 to 0.043): reasoning lowered group loss more with different subject lines than with a shared one.The result was from the pre-specified stage 1 test for GPT-6 Sol; the comparison was descriptive against independent agents at the observed participation rate.Source: §5.1 · p. 37, Figure 2 · p. 39

5.2 Features discovered after data collection

  • Screening about 20 candidate features identified subject-line length and the first post-prefix character’s code point as divisibility-by-3 features associated with participation.The code-point feature coincided with five of the 32 opening colour words, so the data could not separate the feature from those words; associations do not identify how choices were computed.Source: §5.2 · p. 37

5.3 Prospective text runs and their limits

  • In prospective run 1, the Japanese rule effect was 0.43; pre-specified Japanese variance readings were also met in run 2 and with two Japanese prompt wordings in run 3.The pooled estimate across the three Japanese runs was exploratory: 0.37 (95% CI 0.18 to 0.55).Source: §5.3 · p. 38
  • Several pre-specified predictions were not met, including positive Chinese variance and dependence following subject-line rather than prompt language, although weak positive rule effects appeared for Chinese lines and English prompts with Japanese lines.Rule effects were near zero for Korean, German, and English subject lines.Source: §5.3 · p. 38
  • Run 3’s restructured prompt changed several features together, so it did not isolate the effect of removing the closing sentence.The prospective runs used GPT-6 Sol and the same subject-line vocabularies, tested individual choices, and leave generalization to other models or text generators unresolved.Source: §5.3 · p. 38

Supplementary Note 6: Exploratory pilots and the origin of stage 1

Exploratory runs preceded the separate stage 1 study. The pilot and F1 findings informed later study design, while several pilot runs were excluded from analysis or left incomplete.

Exploratory pilots and the origin of stage 1

  • A Japanese-prompt arm found no decision difference between shared and individual reference tokens for Claude Sonnet 5 and Gemini 3.8 Flash.A separate arm in which Claude Opus 5.5 and GPT-6 Sol wrote randomization code was not analysed here.Source: Supplementary Note 6: Exploratory… · p. 40
  • In F1, reasoning increased excess group loss with a shared context note without identifiers (contrast 0.026; exploratory 95% interval 0.005 to 0.039), while the effect vanished with private identifiers (−0.001).F1’s pre-specified gate for proceeding with its original plan was not passed, so stage 1 was designed as a separate confirmatory test.Source: Supplementary Note 6: Exploratory… · p. 40
  • The text features found after stage 1 showed the same pattern retrospectively in F1: participation with reasoning was 0.11, 0.40, 0.47, and 0.69 across feature cells.F1 was not used to discover the features.Source: Supplementary Note 6: Exploratory… · p. 40
  • Before the identifier studies, researchers ran two small GPT-6 Luna exploratory screens on 24 September (424 and 1,816 requests) and an English pilot on 25 September (3,058 of 7,084 planned requests sent).The pilot was halted after a refused request; under a documented amendment, two of its four arms were completed, and unsent requests in the other two were dropped.Source: Supplementary Note 6: Exploratory… · p. 40
Loading 2610.09667v1…