Source-linked AI summary
CLUE: Non-parametric Verification from Experience via Hidden-State Clustering
Zhenwen Liang, Ruosen Li, Yujun Zhou, Linfeng Song, Dian Yu, Xinya Du, Haitao Mi, Dong Yu
TL;DR
Finding the correct answer among many plausible LLM outputs remains a verification challenge, and existing text-based and confidence-based approaches have limitations. CLUE verifies reasoning using hidden-state activation deltas and experience-based success and failure centroids. It performs strongly against judge and confidence baselines and improves reranking across AIME and GPQA.
Problem
Reliable selection is difficult when an LLM produces many plausible answers, while text-based judgments can be misled and confidence signals are limited by poor calibration.
Source: §1 · p. 1Method
CLUE summarizes hidden-state reasoning trajectories by activation deltas and classifies them by distance to success and failure centroids from labeled experience.
Source: §1 · p. 2, §3.1 · p. 4Results
CLUE matches or surpasses LLM-as-a-judge and confidence-based baselines; reranking beats majority voting on AIME and GPQA, including 70.0% top-maj@16 versus 56.7% majority@64 on Nemotron-1.5B AIME 24.
Source: §1 · p. 2, §4.4 · p. 7Takeaways & Limitations
Correctness appears geometrically encoded in hidden states, with cleanly separable representations in RL-fine-tuned models largely absent in their SFT counterparts.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Assessing the quality of Large Language Model (LLM) outputs presents a critical challenge. Previous methods either rely on text-level information (e.g., reward models, majority voting), which can overfit to superficial cues, or on calibrated confidence from token probabilities, which would fail on less-calibrated models. Yet both of these signals are, in fact, partial projections of a richer source of information: the model's internal hidden states. Early layers, closer to token embeddings, preserve semantic and lexical features that underpin text-based judgments, while later layers increasingly align with output logits, embedding confidence-related information. This paper explores hidden states directly as a unified foundation for verification. We show that the correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations. To validate this, we present Clue (Clustering and Experience-based Verification), a deliberately minimalist, non-parametric verifier. With no trainable parameters, CLUE only summarizes each reasoning trace by an hidden state delta and classifies correctness via nearest-centroid distance to ``success'' and ``failure'' clusters formed from past experience. The simplicity of this method highlights the strength of the underlying signal. Empirically, CLUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates, improving both top-1 and majority-vote accuracy across AIME 24/25 and GPQA. As a highlight, on AIME 24 with a 1.5B model, CLUE boosts accuracy from 56.7% (majority@64) to 70.0% (top-maj@16).
1 Introduction
Finding the correct answer among many plausible LLM outputs is a verification challenge. CLUE uses hidden-state reasoning trajectories as a unified verification signal and performs strongly against judge and confidence-based baselines.
1 Introduction
- LLMs can produce many plausible answers, making reliable selection of the correct one a central verification challenge.The challenge is to select the correct answer from dozens of alternatives.Source: §1 · p. 1
- Text-based judges and majority voting can be misled by stylistic artifacts, while confidence-based metrics are fragile on poorly calibrated or smaller models.Trained judges can also be brittle under distribution shift and expensive to retrain for new domains.Source: §1 · pp. 1–2
- Hidden states combine semantic and lexical cues in early layers with confidence-related signals in later layers, and correct and incorrect trajectories separate geometrically.The paper reports this separation across tasks and model scales.Source: §1 · p. 2, Figure 1 · p. 2
- CLUE summarizes each reasoning trace by its activation delta and classifies it by distance to success and failure centroids built from labeled historical trajectories.It uses layer-averaged Euclidean distance and does not require a learned judge.Source: §1 · p. 2
- CLUE matches or surpasses LLM-as-a-judge and confidence-based baselines, with robust generalization reported across tasks and model scales.The paper highlights gains on smaller or less-calibrated models where probability cues are unreliable.Source: §1 · pp. 2–3
2 Related Work
Related work studies hidden-state geometry for latent reasoning and several approaches to test-time scaling. CLUE differs by using activation deltas and experience-derived centroids without a trainable verifier.
2.1 Latent Reasoning and Activation Geometry
- Prior work studies latent reasoning and activation geometry, including intermediate activations that align with output distributions and hidden-state probes for verification.Other work uses semantic clustering or activation directions for robustness, monitoring, and steering.Source: §2.1 · p. 3
2.2 Test Time Scaling
- Test-time scaling includes sampling and voting, sequential refinement, and learned or confidence-based selection, whereas CLUE reranks using success and failure centroids without a trainable verifier.The passage contrasts CLUE with reward models, hidden-state rewards, and internal-confidence filtering.Source: §2.2 · p. 3
3 The CLUE Framework
CLUE treats correctness verification as a geometric comparison between hidden-state reasoning trajectories and centroids formed from labeled experience. Its activation-delta representation supports classification and candidate reranking while keeping the LLM frozen.
3 The CLUE Framework
- CLUE captures systematic differences between correct and incorrect reasoning trajectories through training-free aggregation of activation deltas and class centroids.A new trajectory is verified by its proximity to the success and failure centroids.Source: §3 · p. 3, Figure 2 · p. 4
3.1 Problem Formulation
- A trajectory is a prompt-response pair with a binary correctness label, and the verification function uses hidden states while the LLM remains frozen.The label distinguishes success from failure.Source: §3.1 · p. 4
3.2 CLUE: Verification via Activation-Delta Summaries
- CLUE represents the reasoning transformation as the difference between hidden states at the start and end of the explicit reasoning block.The activation-delta matrix uses hidden states across all layers as the sole verification feature.Source: §3.2 · p. 4
3.3 Centroid Construction and Classification
- CLUE averages labeled trajectory deltas into success and failure centroids, then classifies a new trajectory by its layer-averaged distances to them.The nearer centroid determines the predicted class, without gradient-based optimization.Source: §3.3 · pp. 4–5
3.4 Application to Solution Reranking
- For reranking, CLUE scores each candidate by distance to the success centroid and ranks candidates in ascending score order.The ranking supports top-1 selection or prioritization within majority-vote aggregation.Source: §3.4 · p. 5
3.5 Rationale for a Minimalist, Experience-Based Design
- The minimalist, deterministic aggregation is intended to isolate the hidden-state representation’s contribution and reduce overfitting risk while supporting generalization.The paper presents this as a lightweight complement to text-level and confidence-based signals.Source: §3.5 · p. 5
4 Experiments
The experiments assess CLUE as both a solution classifier and a reranker across mathematical and general-reasoning benchmarks, model scales, and training paradigms. Results show improved reranking accuracy, cross-model verification by RL-trained models, and stronger correctness separation in deeper layers.
4.2 Evaluation Setups and Baselines
- Evaluation measures binary correct/incorrect classification and reranking of 64 candidates, using GPT-4o judges, majority voting, DeepConf, and pass@64 as baselines or reference.Reranking reports top@1 and top-maj@k for k ∈{4, 8, 16}; classification labels come from a rule-based verifier.Source: §4 · p. 5, §4.2 · p. 6
4.1 Datasets and Model Configuration
- The experience set uses 10,000 correct and 10,000 incorrect trajectories sampled from AIME and MATH, with AIME 2024/2025 and GPQA as test sets.The study evaluates Nemotron-1.5B, Polaris-4B, and DeepSeek-R1-0528-Qwen3-8B with reasoning lengths of 16k, 32k, and 64k tokens.Source: §4.1 · p. 6
4.3 Classification Performance
- CLUE maintains a more balanced classification profile than GPT-4o, with TNR up to 87.4% on Nemotron-1.5B and TPR up to 89.5% on Polaris-4B outputs.GPT-4o's TNR falls to 34.4% on Polaris-4B solutions on AIME 2025, reflecting its tendency to misclassify incorrect solutions as correct.Source: §4.3 · p. 7, Table 1 · p. 7
4.4 Reranking for Enhanced Reasoning Accuracy
- On AIME 24, Nemotron-1.5B top-maj@16 reaches 70.0% versus majority@64 56.7%; on GPQA, Polaris-4B reaches 59.6% versus majority voting 56.6%.Across AIME and GPQA, CLUE reranking consistently outperforms majority voting; CLUE also maintains an edge across model scales where confidence cues are poorly calibrated.Source: §4.4 · pp. 7–8, Table 2 · p. 8
4.5 Generalization and the Influence of Training Paradigms
- RL-trained verifiers transfer across models: Nemotron-1.5B raises Deepseek-7B accuracy to 80.0% from a 76.7% baseline, while Polaris-4B raises Qwen3-4B outputs to 83.3%.SFT self-reranking barely matches or can lag majority@64; the authors attribute the difference to RL's direct contrastive supervision for correct and incorrect reasoning.Source: §4.5 · pp. 8–9, Figure 3 · p. 8
4.6 Generalization to Diverse, Non-Mathematical Reasoning
- On WebInstruct-verified, CLUE outperforms GPT-4o for both model sizes: accuracy is 60.4% versus 54.0% for 1.5B, and 59.2% versus 48.1% for 4B.CLUE centroids are computed from the WebInstruct training set.Source: Table 3 · p. 9, §4.6 · p. 9
4.7 Layer-wise Separability Analysis
- PCA projections show success and failure clusters overlapping in shallow layers, separating in middle layers, and forming compact clusters in final layers.The centroid-distance curve rises with depth and typically peaks in the final layers, indicating more explicit, separable correctness signals there.Source: §4.7 · pp. 9–10, Figure 4 · p. 10
5 Conclusion
The conclusion presents LLM reasoning as geometrically structured, with correctness signals accessible in hidden states. CLUE uses this structure for verification and highlights a relationship between training paradigm and representational separability.
5 Conclusion
- CLUE achieves verification performance through simple geometric clustering of past experiences and is described as more robust than LLM judges and confidence-based methods.The framework is non-parametric.Source: §5 · p. 10
- Reinforcement Learning-fine-tuned models develop cleanly separable representations for correct and incorrect reasoning, a property described as largely absent in SFT models.Source: §5 · p. 10
- The authors argue that CLUE’s results support analyzing and shaping internal reasoning, and may motivate lightweight generalizable verifiers and training objectives for representational clarity.Source: §5 · p. 10
A.1 Algorithmic Details
CLUE first builds success and failure reference centroids from labeled reasoning trajectories, then classifies new trajectories by their distance to those references. Both phases use activation-delta matrices while the underlying language model remains frozen.
A.1 Algorithmic Details
- The centroid-aggregation and inference phases operate on activation-delta matrices and use layer-averaged Euclidean distance, with the underlying LLM frozen throughout.Source: §A.1 · p. 14
- The learning phase partitions labeled trajectories, extracts hidden states at reasoning-block boundaries, computes activation deltas, and averages them by class to form success and failure centroids.The centroids serve as geometric references during inference.Source: §A.1 · p. 14
- At inference, CLUE computes a new trajectory’s activation delta, measures its distances to both centroid matrices, and assigns the nearer centroid’s label.The success centroid yields a correct classification; the failure centroid yields an incorrect classification.Source: §A.1 · p. 14