PaperlayerView exact revision on arXiv

Source-linked AI summary

Lingtai: What Concept Geometry Reveals--and Does Not Reveal--About LLM Inference

Jiangang Chen

arXiv:2610.00656v1cs.CL

TL;DR

Lingtai addresses the difficulty of observing the hidden computation of language-model inference, where learned probes require training data and output-level uncertainty measures do not expose intermediate computation. It projects residual states onto named concept anchors and finds uncertainty-linked activity and execution-specific trajectory identity, with both structures sharply bounded.

  • Problem

    Language-model inference hides its intermediate computation; learned probes require training data, while output-level uncertainty measures do not directly expose that computation.

    Source: §1 · p. 1

  • Method

    Lingtai projects residual states onto named concept anchors to expose online activity structure as a lightweight representation.

    Source: §1 · p. 2

  • Results

    Lingtai reveals recurring uncertainty-linked activity and execution-specific trajectory identity, but neither is a general correctness signal under the tested protocol.

    Source: §7 · p. 9

  • Takeaways & Limitations

    These are bounded empirical structures: activity geometry is task-conditioned, and trajectory identity is locally robust but weakly invariant to re-instantiation.

    Source: §5 · p. 9

    The correctness audit covers one code model and 163 replayed problems, so it neither supports a correctness coordinate beyond this protocol nor rules one out.

    Source: §6 · p. 9

Abstract

from arXiv · show

Observing what a large language model computes during autoregressive inference--online and without training probes--remains difficult. We introduce Lingtai, a training-free concept telemetry layer: at each generation step, residual states are projected onto a domain-specific bank of named concept anchors, constructed without labeled concept examples, outcome labels, gradient fitting, or activation-space optimization, producing a structured per-step concept-coordinate signal. Across code generation and grade-school mathematical reasoning, this signal exhibits a robust association with predictive uncertainty: the association survives problem-identity and token-position controls and is not attributable to a single token type, is not explained by a simple correct/incorrect mixture on GSM8K, and is not reproduced by matched random anchors; it is markedly weaker or direction-inconsistent in K-means and PCA projections. Two structures emerge: a recurring uncertainty-linked activity signal whose functional geometry is task-conditioned (distinct activity-entropy shapes on HumanEval, MBPP, and GSM8K), and an execution-specific trajectory identity with strong local inertia but weak re-instantiation invariance--under completion-only elastic alignment, corruption at k=32 (approximately a median quarter of the completion) on the matched re-execution subset still retrieves the archived episode at 62.0%, while a fresh execution retrieves it only 11.7-16.0% of the time. Finally, a matched audit finds no evidence that the scalar concept-activity signal used here supplies a stable correctness coordinate under the tested protocol; we therefore treat correctness as externally supplied. Telemetry adds 0.7-1.6% per-token decode overhead for the 161-anchor code implementation, with unchanged generated tokens.

1 INTRODUCTION

Lingtai addresses the difficulty of observing hidden inference computation with training-free, named concept coordinates. Its measurements track uncertainty across tasks, while their geometry and execution fingerprints have bounded scope and scalar activity does not provide a stable correctness coordinate under the tested protocol.

Approach

  • Lingtai projects residual states onto named concept anchors without labeled concept examples, outcome labels, or activation-space fitting, producing per-step concept activity.The anchors are discovered from domain terminology, with fixed arithmetic primitives additionally included in the math bank.Source: §1 · p. 1

Findings

  • Across four benchmark–model configurations, concept activity has a positive paired log-ratio advantage over random anchors in 40/50 to 50/50 problems, with bootstrap confidence intervals excluding zero.The association survives problem-identity and token-position controls, is not attributable to one token type, and is not reproduced by matched random anchors; K-means and PCA retain less of it with direction-inconsistent results.Source: §1 · p. 2
  • Uncertainty-linked activity recurs across tasks, but its functional form differs, and execution identity is locally robust yet weakly invariant to re-instantiation.The scalar activity signal supplies no stable correctness coordinate under the tested protocol, so correctness is treated as externally supplied.Source: §1 · p. 2

Contribution

  • The paper frames Lingtai as a training-free measurement layer that exposes activity structure and execution fingerprints while explicitly characterizing their limits.The 161-anchor code implementation adds 0.7–1.6% per-token decode overhead, with generated tokens unchanged.Source: §1 · p. 2

2 RELATED WORK

The paper distinguishes Lingtai from learned probes, sparse dictionaries, concept explanations, output uncertainty measures, and correctness detectors. Its contribution is a training-free named-coordinate measurement layer, not a sparse decomposition, intervention, or correctness verifier.

Interpreting internal representations

  • Linear probes require per-property training and may supply the structure, while sparse autoencoders require a model- and layer-specific fitting stage.The authors treat SAEs as a learned-feature alternative rather than a dimension-matched baseline, which would require training them for these models.Source: §2 · pp. 2–3

Concept-based explanations

  • Unlike labeled concept-direction methods such as TCAV, Lingtai constructs anchors without labeled concept examples and uses coordinates for online telemetry, not attribution or control.The cited methods include TCAV (Kim et al., 2018), FastCAV (Schmalwasser et al., 2025), and representation engineering (Zou et al., 2023).Source: §2 · p. 3

Uncertainty from outputs

  • Output-distribution uncertainty measures operate on outputs, whereas Lingtai reads residual states and only partially overlaps predictive entropy (r = 0.518 on HumanEval/Qwen2.5-Coder-7B-Instruct).The passage names calibration (Guo et al., 2017), self-evaluation (Kadavath et al., 2022), and semantic-entropy variants (Kuhn et al., 2023).Source: §2 · p. 3

Execution identity and fingerprints

  • Lingtai’s fingerprint is a per-episode trajectory signature, locally robust under corruption but weakly invariant to re-instantiation—not a model identifier or semantic embedding.The passage contrasts it with output memorization and extraction (Carlini et al., 2021) and global representation-similarity indices (Kornblith et al., 2019).Source: §2 · p. 3

Correctness signals from internal states

  • Correctness and hallucination methods produce verifiers or risk estimators, whereas Lingtai measures named coordinates without concept or outcome labels and makes no claim to outperform intrinsic statistics.The passage notes that raw hidden variance tracks entropy more strongly than the corpus anchors (Table 1).Source: §2 · p. 3

3 METHOD

The method builds frozen, domain-specific concept anchors and projects residual states onto them to produce online telemetry. Separate protocols evaluate uncertainty association, execution-trajectory retrieval, and runtime cost across specified models and benchmarks.

3.1 CONCEPT ANCHORS (TRAINING-FREE, DOMAIN-SPECIFIC)

  • The anchor bank is built from target-domain terminology using a co-occurrence network, with each concept title encoded by the target model to define its residual-state center.The bank is a fixed, dense, non-orthogonal measurement frame, with 161 code and 1552 math anchors; the math bank adds 55 fixed numeric/operator primitives.Source: §3.1 · pp. 3–4, Figure 1 · p. 4

3.2 CONCEPT TELEMETRY

  • At each processed token position, the residual state is compared with anchors through RBF similarity to form the telemetry vector, from which activity variance and effective participation are computed.For episode retrieval, trajectories are sequences of telemetry vectors and the primary similarity measure subtracts the frozen-bank mean before pooling.Source: §3.2 · p. 4

3.3 AUDIT PROTOCOL FOR EXTERNAL VALIDITY

  • External-validity audits use problems as the inferential unit and compare within-problem activity deciles against predictive entropy under matched controls.Controls cover problem identity, token position, token type, and GSM8K outcome strata; random anchors, K-means, and PCA are matched baselines where applicable.Source: §3.3 · p. 4

3.4 EXECUTION-FINGERPRINT PROTOCOL

  • Execution fingerprints are evaluated against a frozen bank of 402 MBPP episodes using centered self-margin and top-1 retrieval.The protocol contrasts corruption of a fixed execution with fresh stochastic re-execution and paraphrases of the same problem.Source: §3.4 · p. 5

3.5 COST

  • The cost test compares baseline decoding with a telemetry hook computing the last-token projection and activity variance, while verifying token-for-token identity.It measures prefill, per-token decode, and throughput with warmup and three repetitions.Source: §3.5 · p. 5

3.6 CONFIGURATIONS

  • The uncertainty audit covers HumanEval and MBPP with Qwen2.5-Coder-7B-Instruct, HumanEval with Qwen2.5-Coder-14B, and GSM8K with Phi-2.Fingerprint experiments use OpenCoder-1.5B-Base, while the matched correctness audit uses 163 replayed problems; trajectories are extracted by teacher-forcing recorded completions.Source: §3.6 · p. 5

4 RESULTS

Concept activity is associated with predictive uncertainty across the tested configurations, but its relationship to uncertainty varies by task. Execution fingerprints are locally robust yet weakly invariant across re-executions, and the matched audit finds no stable correctness coordinate under the tested protocol.

4.1 EXTERNAL VALIDITY AGAINST PREDICTIVE ENTROPY, WITH MATCHED BASELINES

  • Across four configurations, CNCE beats random anchors in 40–50/50 problems; its uncertainty association survives controls, while K-means and PCA are weaker or inconsistent.The audit reports weaker external association for K-means and PCA than telemetry, with some PCA configurations near zero or negative; the table covers HumanEval/Qwen2.5-Coder-7B-Instruct using predictive entropy.Source: §4.1 · pp. 5–6, Table 1 · p. 6

4.2 THE ACTIVITY–ENTROPY RELATION IS TASK-CONDITIONED

  • Within-problem activity–entropy curves differ by task: HumanEval is monotone (d9/d0 = 7.1×), MBPP U-shaped (2.8×), and GSM8K slow-then-steep (12.0×).The association replicates in direction, but not in functional form; the GSM8K comparison is cross-domain and qualitative, while HumanEval and MBPP share a matched model and bank.Source: §4.2 · pp. 6–7, Figure 2 · p. 6

4.3 WHAT THE ACTIVITY SIGNAL CONSISTS OF: COMMON-MODE AND CENTERED STRUCTURE

  • The raw activity profile is common-mode dominated, while centered-profile dispersion is higher at high-entropy tokens across all three benchmarks.Effective participation is near the full anchor dimension in both domains; centered-profile participation rises with entropy in GSM8K but not consistently in the two code configurations.Source: §4.3 · p. 7

4.4 EXECUTION FINGERPRINTS: LOCALLY ROBUST, RE-INSTANTIATION BRITTLE

  • Execution fingerprints retain top-1 retrieval under local corruption but retrieve fresh executions poorly; identifier renames show no special invariance.At k = 128, replacing a median 25% of completion tokens leaves 74–77% top-1 retrieval; fresh re-execution retrieval is 8.5%/7.4%, and consistent renames retain about 99% top-1 retrieval, like arbitrary replacements.Source: §4.4 · pp. 7–8, Figure 3 · p. 7

4.5 NO STABLE CORRECTNESS COORDINATE FROM SCALAR CONCEPT ACTIVITY UNDER THE MATCHED PROTOCOL

  • The position-matched audit finds no evidence that scalar concept activity provides a stable correctness coordinate under the tested protocol.Kvar AUROC falls to 0.494 at synchronization, and it does not exceed prompt-derived baselines in the audit.Source: §4.5 · p. 8, Figure 4 · p. 8

4.6 COST

  • Telemetry adds 0.71%/1.57% per-token decode time on Qwen2.5-Coder-7B-Instruct and OpenCoder-1.5B-Base, with token-identical outputs.The hook adds about 1 MB peak memory; the math bank was not profiled.Source: §4.6 · p. 8

5 DISCUSSION

Lingtai reveals recurring uncertainty-linked activity and execution-specific trajectory structure, but both are sharply bounded rather than universal. The signal is not presented as a correctness detector; the discussion instead sketches scope-bound incident memory as an architectural implication.

5 DISCUSSION

  • The uncertainty-linked activity recurs across domains, but its functional geometry and coordinate basis are task- and domain-conditioned.The paper treats these as bounded empirical structures, not universal properties of LLM inference.Source: §5 · p. 9
  • Execution trajectories show strong local inertia but weak re-instantiation invariance, even under alignment-tolerant rescoring.The telemetry also exhibits local corruption tolerance that exceeds its invariance across fresh executions.Source: §5 · p. 9
  • Lingtai is framed as measurement, not classification: correctness is externally supplied, with no stable correctness advantage under the fair protocol.The discussion does not propose a detector.Source: §5 · p. 9
  • Scope-bound incident memory could bind an external outcome, such as a test verdict or human label, to its generating episode and retrieve it for similar episodes.The proposed implication is not a built system and would retrieve a labeled failure trajectory without generalizing the mistake.Source: §5 · p. 9

6 LIMITATIONS

The limitations bound the evidence by model, task, protocol, and implementation scope. In particular, the correctness audit tests only scalar Kvar in one code model and does not rule out correctness information under other protocols.

6 LIMITATIONS

  • Fingerprint evidence is limited to OpenCoder-1.5B-Base on MBPP-style episodes; the paraphrase set is small and the main analysis uses index-aligned similarity.An elastic-alignment stress test preserves the reported contrast, while cross-layer sweeps remain future work.Source: §6 · p. 9
  • The centered-dispersion analysis is exploratory, lacks held-out confirmation, controls position but not token type, and has a task-dependent participation component.The raw-support analysis is negative.Source: §6 · p. 9
  • The correctness audit covers one code model and 163 replayed problems, tests only scalar Kvar, and neither supports a broader correctness coordinate nor rules one out under other protocols.Directional or subspace structure was not tested.Source: §6 · p. 9
  • The concept banks are domain-specific, and the cost audit covers only the 161-anchor code bank; Panel A uses a single perturbation seed.Cross-domain claims are directional replications, not numerical comparisons.Source: §6 · p. 9

7 CONCLUSION

Lingtai exposes uncertainty-linked activity and execution-specific trajectory identity, but the former is task-conditioned and the latter weakly invariant to re-instantiation. Under the tested protocol, scalar concept activity supplies no stable correctness coordinate.

7 CONCLUSION

  • Lingtai identifies an uncertainty-linked activity signal and an execution-specific trajectory identity, neither of which is a general correctness signal.Under the tested protocol, scalar concept activity supplies no stable correctness coordinate.Source: §7 · p. 9
  • The execution fingerprint recognizes a specific past generation and is locally robust to corruption but brittle to re-instantiation.The work reports no evidence of cross-session identification, while flagging fingerprinting as privacy-relevant.Source: §7 · p. 9
  • Experiments use public benchmarks and open-weight models, and the supplementary package includes protocols, frozen artifacts, configuration details, and reproduction code.The work uses no human subjects or personal data.Source: §7 · p. 9

A REPLICATION DETAILS

The replication details report task-dependent participation patterns and low natural re-execution retrieval rates. Table 3 is identified as reporting direction replication across configurations.

A REPLICATION DETAILS

  • Table 3 reports direction replication across configurations.Source: A REPLICATION DETAILS · p. 12
  • At high entropy, centered effective participation rises in 49/50 GSM8K problems, versus 42%/26% in the two code configurations; the MBPP CI contains zero and HumanEval is below zero.The statistic is reported as a per-problem mean within entropy halves, with token positions restricted to the pooled interquartile range.Source: A REPLICATION DETAILS · p. 12
  • 8.5% [3.8, 16.1] is top-1 retrieval for a fresh sample with seed 42 (n = 94), while 3.3% [0.1, 17.2] is retrieval for paraphrases (1/30 in each of seeds 42 and 7; n = 30 per seed).For fresh samples, seed 7 gives 7.4% [3.1, 14.7]; under the raw half-prefix metric, fresh-sample rates are 21.3% [13.5, 30.9] for seed 42 and 10.6% [5.2, 18.7] for seed 7.Source: A REPLICATION DETAILS · p. 12

B HARDENING CONTROLS AND DEFINITIONS

The appendix tests sensitivity to analysis choices and confounds, defines the activity and fingerprint measures, and bounds comparison and reproducibility claims.

Definitions

  • Kvar is variance across concept activities; predictive entropy is the external criterion, and dlog pairs within-problem high/low Kvar contrasts against random anchors.Entropy is computed from the next-token distribution; problem-level confidence intervals use paired bootstraps over problems.Source: B HARDENING CONTROLS AND DEFINITIONS · pp. 12–13
  • A trajectory is a sequence of telemetry vectors; centered-prefix cosine against the frozen episode bank defines similarity, while divergence is an alignment index, not semantic error onset.Replay fidelity measures the fraction of teacher-forced greedy argmax positions matching the recorded completion; one of 164 runs below 0.8 was excluded.Source: B HARDENING CONTROLS AND DEFINITIONS · p. 13

Fingerprint controls

  • Under completion-only DTW, k=32 corruption retrieves the archived episode at 62.0%, versus 11.7–16.0% for fresh re-execution.The matched-subset audit places k=32 near a median quarter of completions and finds the gap persists under alignment-tolerant scoring.Source: B HARDENING CONTROLS AND DEFINITIONS · p. 13

Baseline sensitivity

  • Across ten random banks, positive-problem counts range 42–50/50, while token-level ρ averages 0.032.Mean dlog is 0.811, but the token-level ρ range is [−0.162, 0.147].Source: B HARDENING CONTROLS AND DEFINITIONS · p. 13

Sensitivity analyses

Confound controls

  • Controlling token log-frequency, position, and problem leaves the Kvar–entropy association intact: partial ρ is 0.323 after adding problem control.The per-problem mean partial ρ controlling frequency and position is 0.275 [0.233, 0.319].Source: B HARDENING CONTROLS AND DEFINITIONS · p. 14

Sensitivity analyses

  • The PCA sign-and-rotation-invariant audit changes token ρ by at most 0.06 and AUROC by at most 0.05, without changing its weak external validity.A scale-free capture-ratio variant is weaker still.Source: B HARDENING CONTROLS AND DEFINITIONS · p. 14

Scope and reproducibility

  • Anchor audits found no substantive benchmark-text overlap; no numerical SAE comparison was run, and reported math results use frozen anchors distinct from the revised deterministic rebuild.The overhead protocol measured the 161-anchor code bank, not the 1,552-anchor math bank.Source: B HARDENING CONTROLS AND DEFINITIONS · pp. 14–15
Loading 2610.00656v1…