PaperlayerView exact revision on arXiv

Source-linked AI summary

Consistency Without Alignment: Item-Sensitive Language Models Indistinguishable From Random

Cris Huynh

arXiv:2609.00576v1cs.AIcs.CL

TL;DR

Item-sensitivity is widely used as evidence that models perform the intended task, but the paper asks whether input dependence alone establishes alignment. It tests this in a forced-choice signalling task with independently computable fit, posterior, and random references, finding that item-sensitivity can coexist with choices indistinguishable from random. The authors argue that such metrics need an independent reference for the quantity of interest.

  • Problem

    Item-sensitivity is widely used as evidence of task competence, although it establishes only that outputs depend on inputs rather than on a model’s output prior.

    Source: §I · p. 1

  • Method

    The paper evaluates language-model choices in a forced-choice signalling task where fit-maximising, posterior-maximising, and random-choice references are computable.

    Source: §I · p. 1

  • Results

    21 of 21 model-by-rule cells are item-sensitive, but 8 are indistinguishable from random and 5 score worse than random at describing the target.

    Source: §I · p. 1

  • Takeaways & Limitations

    Item-sensitivity evidence is not sufficient to establish alignment with the intended quantity without an independent reference.

    Source: §I · p. 1

    The Bayesian reference assumes a literal listener; models or people reasoning about how listeners interpret their choices need not converge on it.

    Source: §V · p. 10

Abstract

from arXiv · show

Item-sensitivity, defined as whether a model's choice depends on the specific input rather than on its own output prior, is widely reported as evidence of task competence. We show this evidence is necessary but not sufficient using a forced-choice signalling task abstracted from the board game Deception: Murder in Hong Kong. In this environment, the reference points against which a coordinate should be judged (a fit-maximising strategy, a posterior-maximising strategy, and uniform random selection) are all computable in closed form. Across seven language models, two model families, a post-training ablation, and three independent scoring rules, every one of 21 model-by-rule cells is reliably item-sensitive. Yet 8 of those 21 cells are not statistically distinguishable from a chooser that ignores the item and selects at random, and 5 score worse than random at describing the target. Item-sensitivity and distance from random correlate at only r = 0.30. We call this consistency without alignment and argue it generalises to any evaluation that relies on item-sensitivity, permutation consistency, or self-consistency without an independent reference for the measured quantity. We further find that a literal-similarity baseline with no pragmatics outperforms most tested language models, that adding a pragmatic layer over two baseline similarity sources moves choosers toward random rather than toward the Bayesian reference, and that a standard labelled multiple-choice format carries no measurable content signal here. All results represent the model side of a pre-registered instrument; a matched human condition is designed and piloted but not yet collected.

I. INTRODUCTION

Item-sensitivity is widely treated as evidence of task competence, but reliable responses to inputs do not establish alignment with the quantity an evaluation intends to measure. Using a signalling task with independently computable reference points, the paper shows that sensitivity and alignment can diverge.

I. INTRODUCTION

  • All 21 model-by-rule cells are item-sensitive, yet 8 are indistinguishable from random and 5 score worse than random at describing the target.Item-sensitivity and distance from random correlate at only r = 0.30.Source: §I · p. 1
  • Item-sensitivity metrics alone lack a reference for whether a model responds to the feature the experimenter intends to measure.The paper calls this problem consistency without alignment and argues that demonstrating it requires an independently computable reference.Source: §I · p. 1
  • The task makes fit-maximising, posterior-maximising, and uniform-random choices computable as independent references for evaluating model choices.It is a forced-choice signalling task abstracted from Deception: Murder in Hong Kong.Source: §I · p. 1
  • A literal cosine-similarity baseline without pragmatics outperforms most tested language models on the measured axis.Adding an RSA-style pragmatic layer over two similarity-based baselines moves them toward random rather than the Bayesian reference.Source: §I · p. 2

A. Generalization and similarity

  • The fit function follows Shepard’s exponential generalization law, while Nosofsky’s GCM motivates reporting both exponential and Gaussian similarity forms.The paper treats the decay family as a reported dimension rather than selecting one form in advance.Source: A. Generalization and similarity · p. 2

B. Rational speech acts, and what is and is not ours

  • The study’s narrower contribution is an exactly computable setting for measuring which criterion a population follows, rather than a new discovery about the fit–posterior trade-off.The paper describes the relationship between fit-maximising and posterior-maximising choices as an identity of its construction, not an empirical finding.Source: B. Rational speech acts, and what is… · p. 2

C. Concept norms and similarity data

The study builds its semantic inputs from human-rated concepts and similarity data, with additional association and symbolic-knowledge sources. Its design fixes the instrument before model runs and uses a closed signalling task that permits exact references.

C. Concept norms and similarity data

  • THINGS supplies concepts, THINGSplus supplies property ratings, and SPoSE, fit to odd-one-out judgments, provides a human-derived similarity space.Because SPoSE compresses human judgments rather than language-model output, it can define an independent variable without making the analysis circular.Source: C. Concept norms and similarity data · p. 2

D. Word association

  • The SWOW cue list’s free-association construction is the measured mechanism behind the vocabulary’s natural-kind retention bias.The paper attributes the bias to how the cue list was built, not merely to a plausible explanation.Source: D. Word association · p. 2

E. Covariates and symbolic knowledge

F. Human-model comparison and cooperative word games

  • The planned human condition uses the same stimuli and choice coordinate as the models, following a shared-instrument comparison tradition.The matched human condition is designed but not yet collected, so the reported results are model-side.Source: F. Human-model comparison and… · p. 2, What is pre-registered and what is not · p. 3
  • Unlike Codenames’ effectively open clue vocabulary, this study’s fixed menu makes salience, Bayesian, and random references exactly computable.Prior Cultural Codes work models sociocultural priors and game context to predict clue-giving and guessing [14]; this study instead uses a homogeneous literal-listener account.Source: F. Human-model comparison and… · pp. 2–3

What is pre-registered and what is not

  • The fit function, item set, scoring rule, and sensitivity battery were fixed in a preregistered instrument before any model was run.Reported model-side measurements are post hoc against that fixed instrument.Source: What is pre-registered and what is not · p. 3

A. Setting

The paper uses a fixed-menu signalling task to study choices among options that were not designed to express the hidden target. It builds the menu and fit measure from human-derived concept ratings and defines fit to preserve differences in how well options describe target pairs.

A. Setting

  • The task models forced lossy encoding: a sender communicates a hidden pair of concepts by choosing one option from a fixed menu not designed for that message.Each item presents M candidate Means, C candidate Clues, and one categorical tile; the hypothesis space contains every M × C pairing.Source: A. Setting · p. 3

B. Concept vocabulary

  • The vocabulary uses a common eligible concept universe so conditions are compared on the same subset, while the sample retains a measured bias toward natural kinds.Concepts come from THINGS and property norms from THINGSplus; the paper traces the bias to SWOW cue-list construction.Source: B. Concept vocabulary · p. 3

C. Tiles

  • Property tiles provide ratings across concepts, and fixed instrument-based bin boundaries preserve a menu not tailored to the message distribution.Four selected dimensions are manmade, size, hold, and moves; unequal bin widths would distort the width-normalized fit measure.Source: C. Tiles · pp. 3–4

D. Fit

  • The exponential fit uses the instrument-defined option width and preserves straddle magnitude; pair fit is the minimum across the two target concepts.The minimum keeps fit positive, preventing a zero-normalization sum or a uniform fallback distribution.Source: D. Fit · p. 4

E. Oracle

The oracle supplies a posterior-based choice reference under a uniform prior and a literal-listener assumption. Items are selected by item properties, then balanced between conflict and control.

E. Oracle

  • The oracle enumerates the exact posterior over 100 hypotheses under a uniform prior and resolves exact ties by posterior, then fit.It represents an S1 speaker addressing a literal listener who does not reason about the speaker’s strategy.Source: E. Oracle · p. 4
  • Fit-maximising choices maximise target-fit numerators, whereas posterior-maximising choices maximise ratios normalized by fit across all hypotheses.They diverge when options vary in how well they fit distractors as well as the target.Source: E. Oracle · p. 4

F. Items

  • The final item set contains 1,000 tile-balanced items, half conflict and half control, selected from 200,000 candidates without using model performance.Source: F. Items · p. 4
  • Decision conflict is when the target-fit-maximising option differs from the listener-posterior-maximising option, with rates varying by tile.Conflict rates range from 3.78 percent on manmade to 37.82 percent on size.Source: F. Items · p. 4

1) Definitions:

The section defines how options are evaluated for target fit and posterior discrimination, then specifies how items are presented and model choices are scored. The three scoring rules can disagree, so findings are recomputed under each.

1) Definitions:

  • The primary dependent variable uses two item-normalised axes: fit to the target and posterior probability of the target given the option.By construction, ofit has fit norm 1 and obayes has post norm 1.Source: 1) Definitions · p. 4
  • Fit cost is the target fit sacrificed when choosing the posterior-maximising option to better discriminate among hypotheses.It is defined as f(ofit, h*) − f(obayes, h*) and serves as the item-difficulty variable and divergence-analysis x-axis.Source: 1) Definitions · p. 5

G. Rendering

  • Concepts appear as bare words, candidate lists are disjoint, option text excludes listed concept names, and option order is randomised across two stored permutations.Glosses were removed after a leak audit found 206 leaking occurrences across 174 items.Source: G. Rendering · p. 5

H. Scoring

  • PMI-normalised scoring subtracts each option’s log-probability under a neutral prompt from its log-probability under the item prompt.The neutral prompt retains the rendered menu and order while stripping item content; options are tokenised in place.Source: H. Scoring · p. 5
  • The scoring rules balance length and predictability concerns: unnormalised sums mechanically favour short options, while per-token means favour predictable ones.Ties are scored as the mean coordinate over tied options, treating them as indifference rather than using an array-order tiebreak.Source: H. Scoring · p. 5
  • The three scoring rules agree on the chosen option only 58.0 percent and 63.4 percent of the time, so findings are recomputed under all three.PMI is primary because it was fixed in advance; the unnormalised sum and per-token mean are also computed.Source: H. Scoring · p. 5

1) The random reference:

The random reference, interval procedure, response format, and model comparisons are specified to make the evaluation interpretable. The design also limits which cross-model comparisons are valid.

1) The random reference:

  • Uniform random selection averages fit norm 0.3985 (SD 0.0294) and post norm 0.4593 (SD 0.0617) across the 125 size conflict items.The spread is reported because a single-point reference could be mistaken for an exact value.Source: 1) The random reference · p. 5

2) Intervals:

  • The 95 percent percentile bootstrap uses 4,000 resamples with items, not paired permutations, as the resampling unit.Model-versus-random comparisons resample paired per-item differences; counts across 21 cells also report Bonferroni-corrected results.Source: 2) Intervals · p. 5

I. Response format

  • The analysis uses format V, where the answer repeats the option text; labelled format L is retired as a measurement condition.The two prompt formats were preregistered because forced-choice answers can flip on trivial relabelling.Source: §I · p. 5

J. Models and environment

  • The model set includes four Qwen3 sizes, base-model post-training ablations, and OLMo-2-7B-Instruct as a cross-family control.The size ladder varies parameter count while holding the training recipe constant; the cross-family control was selected for open pretraining data.Source: J. Models and environment · pp. 5–6
  • Runs use fp16 throughout, while the primary size tile is scored at batch size 1 to avoid padding and batch effects.An equivalent fp32 check collapses the diagnostic difference to 1.9×10^-6; fp32 is unavailable for the 8B model under the stated memory limit.Source: J. Models and environment · p. 6
  • Cross-family comparisons are limited to choice-based statistics, not raw log-probability magnitudes, because the control uses a different tokenizer.Base models use their instruct sibling’s chat template, with the stated one-directional format confound.Source: J. Models and environment · p. 6

K. Conditions

Across the primary size-conflict items, every model-by-scoring-rule cell shows positive item-sensitivity. The study compares similarity, language-model, pragmatic, Bayesian, and human conditions.

K. Conditions

  • The eight conditions include embedding cosine, SWOW association, ConceptNet path scoring, two zero-shot prompts, an RSA layer, a Bayesian oracle, and humans.The first three hold aggregation constant to isolate the similarity source.Source: K. Conditions · p. 6

IV. RESULTS

  • Unless stated otherwise, results use the 125 size conflict items, and every figure reports all three scoring rules with means and medians.Source: §IV · p. 6

A. Models respond to items

  • Item-sensitivity is measured as permutation consistency above the model’s own marginal option preference, rather than above a uniform null.A model choosing one option 93 percent of the time can score 0.87 consistency while remaining entirely item-insensitive.Source: A. Models respond to items · pp. 6–7
  • Excess is positive in all 21 model-by-rule cells on the 125 size conflict items, ranging from +0.024 to +0.260.This property holds for the only tile with this result.Source: A. Models respond to items · p. 7

B. Models align with neither criterion

Model choices fall between random and the salience-maximising pole, while several cells are statistically indistinguishable from random or describe the target worse than random. Positive item-sensitivity therefore does not establish alignment with the measured criterion.

B. Models align with neither criterion

  • The raw post norm is primary because the 0.309-wide interval makes per-item salience-to-Bayes fractions unstable when an item’s span is small.Source: B. Models align with neither criterion · p. 7
  • All 21 model-by-rule cells fall between random and the salience pole, from 0.463 to 0.670; none reaches the salience pole or approaches the Bayes pole.The random reference is 0.459 and the salience pole is 0.691 on the post norm axis.Source: B. Models align with neither criterion · p. 7
  • Eight of 21 cells have intervals including zero against random, with paired differences from +0.004 to +0.046; 11 are not distinguishable after Bonferroni correction.These are precision statements, not claims of equality; at 125 items, intervals are roughly ±0.06 wide.Source: B. Models align with neither criterion · p. 7
  • Five cells fall below the fit-axis random reference of 0.399: control/PMI 0.239, 8B base/sum 0.231, 8B base/token mean 0.179, 1.7B base/token mean 0.225, and 0.6B instruct/PMI 0.389.These choosers describe the target worse than a coin flip would.Source: B. Models align with neither criterion · p. 7
  • The 8B instruct model has the highest median, 0.74 to 0.86 across the three rules, but this is one rung differing from three, not a scale claim.Mean exceeds median in 7 of 21 cells and median exceeds mean in 14; no cell has even half its items agreeing with ofit on both permutations.Source: B. Models align with neither criterion · p. 7

C. Consistency is not alignment

  • All 21 cells have positive excess, including eight statistically random-level cells; across cells, excess and distance from random correlate at only r = +0.30.The 8B base model under the unnormalised sum has excess +0.216 yet lies +0.022 from random with an interval spanning zero.Source: C. Consistency is not alignment · p. 7
  • Item-sensitivity and self-consistency are necessary but insufficient evidence of task competence; evaluating the quantity of interest requires an independent reference.A model may consistently track an item feature that the experimenter does not care about.Source: C. Consistency is not alignment · p. 7

D. Achieved fit is the discriminating axis

The achieved-fit coordinate separates salience and Bayesian strategies, but model fit-cost trends are shallow under the primary rule. A literal cosine baseline exceeds most model cells, while making the listener’s hypothesis space explicit changes choices most for the 8B model.

D. Achieved fit is the discriminating axis

  • Bayes fit falls from 0.972 to 0.403 across fit-cost deciles while salience remains 1.000; models range from fit norm 0.09 to 0.67 against random 0.399 and salience 1.000.The observed model slopes run from −0.43 to +0.25, compared with reference slopes of 0.00 for salience and about −0.90 for Bayes.Source: D. Achieved fit is the discriminating… · p. 8, Figure 3 · p. 9

E. The divergence curve is shallow under the primary rule

  • Under primary PMI, no model’s fit-cost slope excludes zero; identifying the item feature behind size choices requires a manipulation absent from this item set.The 8B instruct model’s predicted change across the observed range is +0.002, against a pre-registered smallest effect of interest of 0.05.Source: E. The divergence curve is shallow… · pp. 8–9

F. The literal baseline outperforms most models

  • Embedding cosine without pragmatics reaches post norm 0.640, 78 percent of the way from random to salience and above 15 of 21 model cells, but cannot support a pragmatic layer.The no-pragmatics condition is the one base condition that cannot support the layer.Source: F. The literal baseline outperforms… · p. 9

G. Making the hypothesis space salient

  • Adding a cue that the listener considers every possible pairing changes choices on roughly one item in six, with agreement of 0.834 overall and the effect concentrated in the 8B model.For the 8B model on manmade, agreement falls to 0.288 and total variation distance between option distributions reaches 0.690.Source: G. Making the hypothesis space salient · p. 9

H. Negative results

The negative findings include structural limits on which baselines can support a pragmatic layer, movement toward random under that layer, and no content signal from labelled multiple choice. Two swept parameters also cannot affect an argmax.

H. Negative results

  • The authors treat negative results as substantive findings and state limitations separately rather than organizing results around only what worked.Source: H. Negative results · p. 9

1) Two conditions are structurally ineligible for a pragmatic layer:

  • Two of four nominated base conditions are structurally ineligible for a pragmatic layer: prompted-model outputs are speaker likelihoods, while cosine values lack a probabilistic interpretation.The authors reject transforms because they either add assumptions or reintroduce a free parameter.Source: 1) Two conditions are structurally… · p. 9

2) The pragmatic layer moves choosers toward random:

  • RSA over SWOW moves post norm from 0.593 to 0.510 and over ConceptNet from 0.651 to 0.472, toward random rather than Bayes; ConceptNet RSA scores 116 items versus 55 for ConceptNet.RSA over ConceptNet ends up indistinguishable from a coin flip.Source: 2) The pragmatic layer moves choosers… · p. 10

4) The labelled response format carries no content signal:

  • Permutation consistency is at or below chance in all 32 model-by-tile-by-label cells, and every canonical distribution sits within 0.10 of uniform, so format L carries no content signal.The two label forms agree on only 64.9 percent of choices.Source: 4) The labelled response format… · p. 10

5) Two parameters cannot affect an argmax:

  • RSA’s rationality exponent cannot reorder a non-negative vector, leaving choices identical across 2,000 base-by-item cells; condition 3’s decay base is likewise inert for argmax.Both parameters remain live for distributional reports, where they sharpen rather than reorder.Source: 5) Two parameters cannot affect an… · p. 10

V. DISCUSSION

The paper’s model-side results support its claim that item-sensitivity alone does not establish task alignment, while the discussion bounds that claim to the evidence collected. The conclusion and accompanying statements also clarify that the human condition remains uncollected and specify what materials are released.

V. DISCUSSION

  • The central claim is supported by model-side results, but no human choices have been collected, so the fit-posterior comparison currently applies only to models.The pre-registered human instrument includes blocks to classify individual strategies and estimate a population curve, since a population mean may describe nobody if strategies vary.Source: §V · p. 10, §VI · p. 11
  • Only the size tile supports item-sensitivity above the marginal null; whether the dissociation generalizes to other tiles remains unknown.The other three tiles do not replicate sensitivity, and their coarser resolution may prevent measuring the strategies.Source: §V · p. 10
  • The Bayesian reference is optimal for an S1 speaker addressing a literal listener, not a general account of pragmatic rationality.A fuller higher-order account is outside the paper’s scope; making the hypothesis space explicit moves one model substantially.Source: §V · p. 10
  • The vocabulary has a measured natural-kind sampling bias linked to the construction of the association norms, and a differently drawn vocabulary could populate the item set differently.The authors have not checked whether changing the vocabulary would affect the results.Source: §V · p. 10
  • A different scoring rule aligning all 21 cells with either reference would change the central claim; the three tested rules did not do so.A different tile with sensitivity but no random-indistinguishable cells would narrow the claim to the tiles tested, while human data could extend or sharpen it without weakening the model-side result.Source: §V · pp. 10–11

VI. CONCLUSION

  • Every one of 21 model-by-rule cells is item-sensitive, yet 8 are indistinguishable from random and 5 score worse than random; the correlation with distance from random is r = +0.30.The authors call this consistency without alignment and argue item-sensitivity needs an independent reference for the quantity of interest.Source: §VI · p. 11

DATA AVAILABILITY

  • The repository includes the final 1,000-item set, rendered prompts, scored choices, decision log, and analysis code, but not redistributed source corpora.SWOW’s license prevents redistribution; the other corpora can be reconstructed from their public sources and the data manifest.Source: DATA AVAILABILITY · p. 11

ETHICS STATEMENT

  • No participants were recruited or participant data reported; the human condition is pre-registered but not yet collected.Two colleagues reviewed instrument wording during development, but their responses are not reported.Source: ETHICS STATEMENT · p. 11

APPENDIX A A MEASURED SAMPLING BIAS IN THE VOCABULARY

The appendices measure a natural-kind sampling bias in the vocabulary, specify an exact condition for ties, and show the instrument’s blank interface renderings.

APPENDIX A A MEASURED SAMPLING BIAS IN THE VOCABULARY

  • Natural kinds are retained at 87.1% versus artifacts at 62.1%, a 25.0-percentage-point gap; single-word conditioning leaves 18.5 points.The single-word restriction explains only 26% of the gap, and the rarest frequency quartile retains a 26.4-point gap.Source: Appendix A · p. 11
  • Associative availability is the measured mechanism: SWOW cue-list membership tracks response frequency, which subsumes corpus frequency in the reported model.Corpus frequency falls from +2.95 (p < 0.001) to +0.47 (p = 0.066) when response frequency is included; pseudo-R2 rises from 0.469 to 0.722.Source: Appendix A · p. 11

APPENDIX B THE EXACT TIE CONDITION

  • Options tie exactly when their likelihood columns over the hypothesis space are proportional; with uniform widths, a common fit factor cancels in posterior normalization.The geometric case of options outside the item’s concept range covers 84.7% of ties, but is sufficient rather than necessary.Source: Appendix B · pp. 11–12

APPENDIX C THE ANNOTATION INTERFACE

  • The interface renderings show the chooser-facing item and, afterward, the tile’s option set and forced-choice question, with no participant data.Source: Appendix C · p. 12, Figure 4 · p. 12
Loading 2609.00576v1…