PaperlayerView exact revision on arXiv

Source-linked AI summary

Correctness Is a Direction: Geometric Answer Selection in Language Models

Marcus Armstrong, Navid Ayoobi, Pradham Mummaleti, Alexander Chulzhanov, Arjun Mukherjee

arXiv:2610.04512v1cs.CL

TL;DR

Language models may encode answer correctness in hidden states even though token-by-token generation does not directly score semantic correctness. The paper extracts a contrastive geometric direction from labeled examples and reports gains over zero-shot log-probability scoring, alongside task-specific geometry.

  • Problem

    Whether answer correctness is recoverable as a geometric direction in single-model hidden states has not been established, despite evidence that other semantic properties are linearly encoded.

    Source: §1 · p. 2

  • Method

    A contrastive mean displacement between correct- and incorrect-answer representations provides a scoring direction from fifty labeled examples, without parameter updates or generation at inference.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    +32.0 percentage points on factual benchmarks and +38.1 to +51.8 percentage points on TruthfulQA over zero-shot log-probability scoring across five models.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Correctness directions for factual reasoning, domain knowledge, and calibrated truthfulness are near-orthogonal, with the degree of separation varying across model architectures.

    Source: §8 · p. 15

    The method is suited to structured candidate selection and consistently formatted single-answer detection, not arbitrary free-form outputs.

    Source: §6.3 · p. 13

Abstract

from arXiv · show

Answer correctness is encoded as a recoverable geometric direction in the hidden states of language models. We show that the mean displacement from incorrect to correct answer representations, computed at approximately 70\% of model depth from fifty labeled examples with no parameter updates, yields a scoring direction that outperforms zero-shot log-probability scoring by up to +32.0 percentage points on factual benchmarks (ARC-Challenge and MMLU) and by +38.1 to +51.8 percentage points on TruthfulQA, across five models spanning 1B to 8B parameters in three architecture families (Llama, Qwen, Gemma). The method requires one forward pass and one dot product per candidate; no generation is performed at inference. Applied as a hallucination detector on individual (question, answer) pairs, the recovered direction achieves 0.693~AUROC versus 0.578 for log-probability scoring. We additionally find that correctness directions for factual reasoning, domain knowledge, and calibrated truthfulness are near-orthogonal in representation space, revealing that language models allocate geometrically independent subspaces to qualitatively distinct notions of correct answer, with architecture-dependent variation in the degree of separation. This structure explains the observed transfer pattern---the direction calibrated on factual questions transfers within task type but not across it---and suggests that LLM calibration failures may reflect a routing problem: the model's internal representation contains more correctness signal than its output behaviour exploits.

1 Introduction

The paper asks whether answer correctness is recoverable as a geometric direction in language-model representations, then uses a contrastive direction to score candidates without parameter updates or generation. It reports gains over zero-shot log-probability scoring and finds that correctness directions vary by task type.

1 Introduction

  • The paper addresses whether answer correctness, beyond abstract semantic properties, is encoded as a recoverable geometric direction in single-model hidden states.Autoregressive generation selects tokens by next-token probability rather than directly measuring semantic correctness.Source: §1 · pp. 1–2
  • A contrastive displacement between correct- and incorrect-answer representations recovers a scoring direction from fifty labeled examples without parameter updates or inference-time generation.Candidate scoring uses one forward pass and a dot product; correct-versus-no-answer displacement instead encodes answer-ness.Source: §1 · p. 2
  • Correctness directions for factual, domain-specific, and calibration-oriented tasks occupy near-orthogonal subspaces, with partial alignment in Gemma-2-2B.This geometry accounts for transfer to same-domain MMLU but not TruthfulQA.Source: §1 · p. 2
  • +32.0 percentage points on factual benchmarks and +38.1 to +51.8 percentage points on TruthfulQA over zero-shot log-probability scoring across five models.TruthfulQA scoring reaches 72.1%–81.2% versus zero-shot baselines of 28.7%–39.5%.Source: §1 · p. 2
  • A logistic-regression probe trained on the same calibration examples matches geometric scoring within 2.1 pp across tested model-task combinations.Direction estimation saturates at approximately fifty labeled examples across tested architectures.Source: §1 · p. 2
  • The optimal extraction depth is approximately 65–70% of model depth across five models with different sizes and layer counts.The paper associates this depth with a representational transition from syntactic to semantic encoding of answer quality.Source: §1 · p. 2
  • The paper characterizes task-orthogonality as a structured set of task-specific correctness directions rather than one geometric concept of correctness.It provides an empirical account of when task transfer succeeds or fails.Source: §1 · p. 2
  • The recovered direction detects hallucinations in individual question-answer pairs at 0.693 AUROC versus 0.578 AUROC for log-probability scoring.This use requires no training beyond the fifty-example calibration step.Source: §1 · p. 2

2 Related Work

Prior work studies linear semantic representations, truth-related directions, hallucination detection, and candidate scoring. This paper distinguishes its untrained mean-displacement approach from trained probes, generation-based methods, and prior truthfulness directions.

Linear representations and probing

  • Unlike probing classifiers, the method trains no classifier and computes a mean displacement over fifty labeled examples, applied with one dot product.The paper places this distinction against linear probing work on syntax, factual knowledge, and sentiment.Source: §2 · p. 3

Directions in hidden states for truth and honesty

  • The contrastive direction targets correct-answer-ness, while standard correct-versus-no-answer displacement encodes answer-ness and is less discriminative.The paper also measures task-direction cosine similarity to geometrically characterize skill-specific transfer failures reported by Orgad et al. [2025].Source: §2 · p. 3

Hallucination detection and confidence estimation

  • The detector uses fifty general question-answer triples and one dot product per candidate, avoiding the extra generations or hallucination-labeled training used by several alternatives.It achieves 0.693 AUROC on binary correct/incorrect classification.Source: §2 · p. 3

Answer selection and scoring

  • The method scores candidates from a single forward pass without generation, outperforming log-probability scoring by up to +32.0 pp on factual benchmarks and +38.1 to +51.8 pp on TruthfulQA.Self-consistency and minimum Bayes risk approaches incur generation costs.Source: §2 · pp. 3–4

3 Geometric Correctness Directions

The section introduces contrastive hidden-state directions for answer selection and examines their discriminative value, optimal extraction depth, and calibration efficiency. Across the tested models, contrastive directions outperform standard directions despite lower signal-to-noise ratios, and useful estimates saturate with about fifty examples.

3.2 Layer Depth and Extraction Point

  • Across five models, the validation-selected optimal extraction depth falls between 56% and 70% of model depth, with the 16-layer Llama model’s optimum at fraction 0.60.Layer selection used a held-out validation split; the authors interpret the depth range as a consistent representational transition relevant to answer quality.Source: §3.2 · p. 5

3.1 The Contrastive Correctness Direction

  • 66.5% (+22.1 pp above zero-shot) is the ARC-Challenge accuracy of Gemma-2-2B’s contrastive direction; contrastive directions outperform standard ones despite lower SNR, except at Llama-3.2-1B’s layer 12.At that Llama layer, standard and contrastive directions have cosine similarity ≈0.99 and accuracies of 58.0% and 53.2%, respectively.Source: §3.1 · p. 4, §3.2 · p. 5

3.3 Calibration Efficiency

  • At N=50, direction estimation has SNR above 7×, and increasing calibration to N=100 does not improve downstream accuracy.The authors attribute the saturation to consistent per-example displacements and the inclusion of noisy borderline questions in larger ARC calibration sets.Source: §3.3 · p. 5, Table 2 · p. 6
  • Calibration uses approximately 250 forward passes and finishes in under five minutes on a consumer GPU for the tested model sizes, without gradient computation.The estimate uses 50 examples and five passes per example.Source: §3.3 · pp. 5–6

4 Task-Specific Geometric Structure

The section finds that correctness directions differ across task types, with task alignment varying by architecture and scale. Transfer is strongest within factual tasks and with task-specific calibration, while coherence tasks can show negative transfer.

4.1 Orthogonality of Correctness Directions

  • Near-orthogonality dominates across task pairs, but alignment varies by model: Llama-3.1-8B has pairwise cosines at or below 0.10, while Gemma-2-2B and Qwen-2.5-7B show partial alignment.The figure compares ARC-Challenge, MMLU-Biology, and TruthfulQA directions across five models.Source: §4.1 · p. 7, Figure 2 · p. 7

4 Task-Specific Geometric Structure

  • The transfer experiments test whether ARC-calibrated directions transfer across six target benchmarks spanning three semantic categories.The section frames task-orthogonality as near-independent task-specific subspaces, with partial exceptions.Source: §4 · p. 6, §4.2 · p. 8

4.2 Transfer Within and Across Task Types

  • The ARC direction transfers positively to factual targets, including +19.7–+20.4 pp on MMLU-Biology, but degrades HellaSwag by −11.0 to −26.7 pp.The authors associate positive transfer with factual-task alignment and interpret negative coherence-task transfer as evidence of a distinct or anti-correlated signal.Source: §4.2 · p. 8
  • TruthfulQA-calibrated directions yield +38.1–+51.8 pp under argmax scoring, while ARC-calibrated directions yield only +4.8–+12.9 pp.The TruthfulQA direction is near-orthogonal to ARC for Qwen and Llama, and partially aligned for Gemma.Source: §4.2 · p. 8

4.3 Multi-Dimensional Correctness

  • The studied tasks distinguish factual correctness, commonsense coherence, and calibrated truthfulness, rather than one universal correctness direction.The authors note that finer divisions within factual tasks and partial overlap across categories remain possible; category boundaries can be estimated from fifty examples per task type.Source: §4.3 · p. 9

5 Geometric Answer Selection

The section evaluates geometric answer scoring across five frozen models, using task-specific calibration and a zero-shot log-probability baseline on the same held-out examples. Gains vary by task, and calibration specificity and model scale qualify the results.

5.1 Experimental Setup

  • The evaluation uses task-specific calibration, generally 50 training examples, and compares geometric scoring with length-normalized zero-shot log-probability on identical held-out examples.For MMLU subjects, the ARC calibration set transfers more effectively than MMLU’s small validation splits.Source: §5 · p. 9, §5.1 · p. 9

5.2 Main Results

  • ARC-Challenge gains are 9.7 pp for Llama-1B, +20.1 pp for Qwen-1.5B, and +22.1 pp for Gemma-2B; the ARC direction also transfers to MMLU.On MMLU, Qwen and Gemma gain +13.5 to +22.4 pp on world history and +19.7 to +20.4 pp on biology, while Llama gains +5.0 to +6.8 pp.Source: §5.2 · p. 10
  • TruthfulQA gains reach +51.7 pp for Llama-3.1-8B (28.7% to 80.4%), while Qwen-7B gains +41.7 pp (39.5% to 81.2%).Zero-shot baselines are near or below chance; the task-calibrated direction encodes the contrast between calibrated and overconfident answers.Source: §5.2 · p. 11
  • HellaSwag gains are model-dependent: Qwen improves by +6.4 pp, while Gemma and Llama fall below zero-shot by −4.7 and −16.4 pp.The authors do not claim geometric scoring as a general-purpose replacement for log-probability on coherence tasks.Source: §5.2 · p. 11

5.3 Analysis

  • On ARC-Challenge, accuracy rises with model size within each family, whereas TruthfulQA gains do not consistently rise with scale.The analysis attributes TruthfulQA gains partly to near-chance zero-shot baselines and notes that ARC-calibrated directions yield only +4.8 to +12.9 pp there.Source: §5.3 · p. 11
  • Improvement magnitude tracks task type: factual tasks gain +5 to +22 pp, while TruthfulQA gains +38 to +52 pp, where zero-shot log-probability is near chance and less discriminative.The paper relates this difference to how well zero-shot log-probability already exploits the model’s internal knowledge.Source: §5.3 · p. 11

6 Hallucination Detection

The section tests whether a calibrated geometric direction can classify individual answers as correct or incorrect, beyond its use for selecting among candidates. It reports stronger AUROC than the baseline but limits the method’s scope to answer formats consistent with calibration.

6.1 Setup

  • The detector classifies each (question, answer) pair by projecting its hidden-state representation onto the ARC-calibrated direction; AUROC evaluates score ordering without choosing a threshold.The test set contains 1,999 pairs: 500 correct and 1,499 incorrect.Source: §6 · p. 11, §6.1 · p. 12

6.2 Results

  • GRM reaches 0.693 AUROC versus 0.578 for zero-shot log-probability on ARC-Challenge answer pairs.Correct and incorrect answers have mean projection scores of +0.0255 and −0.0084, respectively, but their score distributions overlap substantially.Source: §6.2 · pp. 12–13, Figure 4 · p. 12, Table 5 · p. 12
  • Correct answers score higher than all three incorrect alternatives on 69.3% of questions, compared with 33.3% expected at random.The authors caution that overlapping distributions make the method unsuitable as a reliable stand-alone detector.Source: §6.2 · p. 13

6.3 Scope: Structured Candidates vs. Free-form Generation

  • TriviaQA reranking changes exact-match accuracy by −0.7 pp (35.3% versus 36.0%), as long generated answers differ in format from short structured calibration answers.The paper attributes the loss of discriminative meaning to this format mismatch.Source: §6.3 · p. 13
  • The supported boundary is structured candidate selection or single-answer detection with answer formats consistent with calibration, not arbitrary free-form outputs.Format-matched calibration is proposed as a remedy, but its systematic evaluation is left to future work.Source: §6.3 · p. 13

7 Discussion

The discussion argues that roughly fifty examples suffice because direction estimates saturate, and that correctness is represented at semantic layers in task-specific geometric directions. It connects these findings to efficient scoring, while noting architecture-dependent separation and limits on generalization.

7.1 Why Fifty Examples Suffice

  • At N=50, contrastive SNR is 8–15×, and doubling calibration examples to N=100 lowers SNR without improving downstream accuracy.The authors attribute saturation to consistent per-example displacements and the inclusion of less geometrically clear borderline examples in larger sets.Source: §7.1 · p. 13
  • Calibration takes about 250 forward passes and under five minutes on a consumer GPU; candidate scoring then uses a dot product.The same calibration cost applies when recalibrating for a new task or domain, and the calibration set can be discarded afterward.Source: §7.1 · pp. 13–14

7.2 Layer Universality as a Representational Finding

  • Across five models, the optimal extraction depth converges to approximately 65–70% of model depth, where semantic content is integrated before output compression.For Llama-3.2-1B, the selected optimum is fraction 0.60; the other listed model optima range from 0.65 to 0.70.Source: §7.2 · p. 14
  • At Llama-3.2-1B’s selected layer nine, the standard direction is anti-correlated with correctness, while contrastive subtraction cancels the shallow-layer confound.The passage links this inversion to representations still encoding surface-level answer properties rather than answer quality.Source: §7.2 · p. 14

7.3 Multi-Dimensional Correctness in LLM Representations

  • For Qwen and Llama, factual, domain-knowledge, and truthfulness directions are nearly orthogonal (|cos θ| < 0.16), while Gemma and larger Qwen models show partial alignment.The discussion interprets this as independent geometric dimensions for distinct notions of correctness, with architecture and training-data factors potentially interacting; disentangling them requires broader controlled ablations.Source: §7.3 · p. 14
  • The authors suggest TruthfulQA errors may reflect failure to route internal calibration to outputs; geometric scoring yields 72–81% versus 28.7–39.5% for zero-shot log-probability.The authors describe this gap as correctness information lost in generation.Source: §7.3 · p. 14

7.4 Relationship to Probing and Representation Engineering

  • Unlike a trained probe, the method computes a mean displacement that cannot fit features beyond the mean direction, giving it an interpretable geometric meaning.The paper distinguishes its scoring use from Representation Engineering’s use of contrast-pair directions to steer generation; inference requires no modification to the model’s forward pass.Source: §7.4 · pp. 14–15
  • The findings cover instruction-tuned models from three architecture families at 1B–8B parameters; generalization to base models, larger models, and non-English tasks is unknown.The method also requires a fixed candidate set and fifty labeled calibration examples per task.Source: §7.4 · p. 15

8 Conclusion

The paper recovers correctness directions from hidden representations using fifty labeled examples and no parameter updates. The approach improves scoring over zero-shot log-probability, while revealing task-specific geometric structure and motivating further work on broader scoring settings.

8 Conclusion

  • +32.0 percentage points on factual benchmarks and +38.0 to +51.7 percentage points on TruthfulQA: contrastive scoring beats zero-shot log-probability across five models.Inference requires one forward pass per candidate and one dot product, with no generation, sampling, or secondary model.Source: §8 · p. 15
  • Correctness directions for factual reasoning, domain knowledge, and calibrated truthfulness are near-orthogonal across four of five tested models, with partial alignment in Gemma-2-2B and Qwen-2.5-7B.The strongest separation is in Llama-3.1-8B, with max |cos θ|=0.10.Source: §8 · p. 15
  • The authors identify fifty-example calibration as the mechanism by which the scorer adapts to each task’s semantic structure.A logistic-regression probe trained on the same examples matches accuracy within 2.1 pp across tested configurations.Source: §8 · p. 15
  • The authors attribute TruthfulQA’s scoring gap to generation failing to route internal truthfulness signals to outputs, not to models lacking that signal.They report zero-shot log-probability at 28.7–39.5% and geometric scoring at 72.1–81.2%.Source: §8 · p. 15
  • Future directions include a small model trained with a contrastive geometric objective, combining task-specific directions, and extending scoring to format-matched free-form generation.The paper presents free-form generation as a step toward open-ended settings where hallucination detection is practically valuable.Source: §8 · pp. 15–16

A Complete Layer Sweep Results

The appendix describes layer sweeps for ARC-Challenge, with layer selection based on a held-out validation split. It specifies the tested fractions and baseline accuracies, and notes that two fractions map to the same layer in one model.

A Complete Layer Sweep Results

  • The layer sweeps use ARC-Challenge, N=50 calibration examples, n=500 held-out test examples, and layer selection on a 20% validation split.The tested fractions are 0.60, 0.70, 0.75, 0.80, and 0.90 for sub-3B models, with 0.65 and 0.85 added for 7B models.Source: A Complete Layer Sweep Results · p. 18
  • Zero-shot log-probability baselines are Llama 37.7%, Qwen 42.0%, and Gemma 46.0% on the stated ARC-Challenge setup.Source: A Complete Layer Sweep Results · p. 18
  • For the 16-layer Llama-3.2-1B-Instruct sweep, fractions 0.75 and 0.80 both map to layer 12, so their results are identical by construction.Source: Table 6 · p. 18, A Complete Layer Sweep Results · p. 18

B Calibration Size Analysis

The calibration-size analysis reports that increasing from 50 to 100 examples lowers signal-to-noise ratio in every tested configuration, with no systematic accuracy improvement.

B Calibration Size Analysis

C MMLU Subject Transfer: Full Results

This block reports ARC-calibrated direction accuracy by MMLU subject and the GRM detector’s threshold operating behavior.

C MMLU Subject Transfer: Full Results

  • Table 12 reports accuracy by MMLU subject using the ARC-Challenge calibrated direction and each model’s optimal contrastive layer.Zero-shot baselines are model-specific and use identical test sets.Source: C MMLU Subject Transfer: Full Results · p. 19

D Hallucination Detection: Threshold Operating Points

E Implementation Details

The implementation details specify the prompt and candidate-scoring procedure, calibration and inference algorithms, layer-selection context, and computational requirements.

E Implementation Details

  • The fixed prompt is “Question: {Q}\nAnswer: {A}”; hidden states come from its final position, with an empty answer for that calibration condition.Source: E Implementation Details · p. 19
  • Algorithms 1 and 2 formalize the calibration and inference procedures described in Section 3.Source: E Implementation Details · pp. 19, 22
  • Table 14 summarizes model architecture parameters and the absolute layer indices corresponding to optimal extraction fractions.Source: E Implementation Details · p. 19, Table 14 · p. 22
  • Layer sweeps use a 20% validation split, with zero-shot baselines of 45.6% for Llama-3.1-8B-Instruct and 48.0% for Qwen-2.5-7B-Instruct.Source: Table 10 · p. 20, Table 9 · p. 20
  • On one GPU, calibration with N=50 takes about five minutes, and evaluation of 500 examples with four candidates takes about 25 minutes.Inference adds one dot product per candidate; the direction vector is stored as a single float32 tensor.Source: E Implementation Details · p. 20
  • Accuracy shows no systematic benefit beyond N=50, while signal-to-noise ratio consistently decreases as calibration size increases.Source: Table 11 · p. 21
  • Candidate answers are scored by a direction-hidden-state dot product, and the answer with the highest score is selected.Source: E Implementation Details · p. 22

Related papers

Loading 2610.04512v1…