PaperlayerView exact revision on arXiv

Source-linked AI summary

ActivationReasoning: Logical Reasoning in Latent Activation Spaces

Lukas Helff, Ruben Härle, Wolfgang Stammer, Felix Friedrich, Manuel Brack, Antonia Wüst, Hikaru Shindo, Patrick Schramowski, Kristian Kersting

arXiv:2510.18184v3cs.LGcs.AI

TL;DR

LLMs have opaque, entangled activations, while SAE features alone provide limited support for systematic reasoning and control. ActivationReasoning grounds concepts in latent features, maps activations to propositions, and applies logical rules to compose and infer new structures. Across four task settings, it substantially outperforms baselines and supports more transparent, robust, and controllable behavior, though its current implementation depends on imperfect SAE features.

  • Problem

    LLM activations are distributed and entangled, and SAE features remain imperfect and passive, limiting explicit reasoning, interpretability, and control.

  • Method

    ActivationReasoning grounds concepts in latent representations, maps detected activations to propositions, and applies user-defined logical rules to produce enriched activations.

  • Results

    Across four experiments, AR substantially outperforms baselines, including larger instruction-tuned and reasoning models, while maintaining above 93% accuracy across 1–5 hops on PrOntoQA.

  • Takeaways & Limitations

    Latent activations can serve as a substrate for structured, interpretable, auditable reasoning and downstream control across reasoning, abstraction, robustness, and safety tasks.

  • Takeaways & Limitations

    AR currently relies on SAEs whose features may be polysemous, context-dependent, incomplete for abstract notions, and difficult to control during discovery.

Abstract

from arXiv · show

Large language models (LLMs) excel at generating fluent text, but their internal reasoning remains opaque and difficult to control. Sparse autoencoders (SAEs) make hidden activations more interpretable by exposing latent features that often align with human concepts. Yet, these features are fragile and passive, offering no mechanism for systematic reasoning or model control. To address this, we introduce ActivationReasoning (AR), a framework that embeds explicit logical reasoning into the latent space of LLMs. It proceeds in three stages: (1) Finding latent representations, first latent concept representations are identified (e.g., via SAEs) and organized into a dictionary; (2) Activating propositions, at inference time AR detects activating concepts and maps them to logical propositions; and (3)Logical reasoning, applying logical rules over these propositions to infer higher-order structures, compose new concepts, and steer model behavior. We evaluate AR on multi-hop reasoning (PrOntoQA), abstraction and robustness to indirect concept cues (Rail2Country), reasoning over natural and diverse language (ProverQA), and context-sensitive safety (BeaverTails). Across all tasks, AR scales robustly with reasoning complexity, generalizes to abstract and context-sensitive tasks, and transfers across model backbones. These results demonstrate that grounding logical structure in latent activations not only improves transparency but also enables structured reasoning, reliable control, and alignment with desired behaviors, providing a path toward more reliable and auditable AI.

1 INTRODUCTION

ActivationReasoning (AR) embeds explicit logical reasoning into LLM latent activations to address opaque, entangled representations and the lack of propositional structure. It grounds concepts in SAE features, maps activations to propositions, applies rules, and evaluates the resulting control and reasoning capabilities across complementary tasks.

  • Motivation: AR addresses superposed, entangled LLM representations and the absence of explicit propositional structure, which limit interpretability, systematic reasoning, and rule enforcement.SAEs provide a foothold by surfacing features that often approximate discrete, human-recognizable concepts, but remain imperfect and passive.
  • ActivationReasoning: AR identifies SAE-grounded concepts, detects their token-level activations as propositions, and composes them with user-defined logical rules into higher-order structures.The three-stage pipeline produces an enriched activation representation for downstream transparency and control.
  • Capabilities: AR can recover composite concepts, disambiguate polysemous features, and support robustness, safety, and multi-hop reasoning through latent-space logical inference.Examples include composing Golden Gate Bridge from Bridge, San Francisco, and USA, and applying logical rules to indirect or noisy cues.
  • Evaluation: Across PrOntoQA, Rail2Country, ProverQA, and BeaverTails, AR substantially outperforms baselines while addressing deductive reasoning, abstract cues, natural language variability, and context-sensitive safety.The evaluation spans multi-hop reasoning, meta-level descriptions, naturalized logical tasks, and real-world safety concepts.
  • Contributions: The paper contributes AR, the Rail2Country benchmark, and evidence that latent activations can serve as a substrate for structured reasoning, transparency, robustness, and control.These contributions connect latent concept representations with explicit composition and rule-based inference.

2 RELATED WORK

AR builds on interpretability, neuro-symbolic reasoning, and safety research by adding an explicit logical layer over latent concept activations. This design aims to make reasoning within large models more compositional, interpretable, auditable, and controllable.

  • Neural Concept-level Interpretability: SAE-based interpretability surfaces semantically meaningful features, while AR extends concept-level analysis by composing latent propositions into higher-level abstractions.Prior approaches commonly treat concepts in isolation and provide limited mechanisms for composition or polysemy disambiguation.
  • Compositionality and Reasoning: Unlike differentiable neuro-symbolic approaches requiring end-to-end training, AR imposes an explicit logical layer over SAE-based features for interpretable latent-space inference.Rules over latent features bridge neural activations and symbolic reasoning without requiring differentiable end-to-end training.
  • Applications in Alignment, Safety, and Knowledge Integration: AR extends latent-feature steering and safety monitoring by applying explicit logical constraints and abstractions directly within the latent space.The framework targets safe model steering, interpretable classification, and multi-hop reasoning with transparency and controllability.

3 ACTIVATIONREASONING

ActivationReasoning (AR) turns latent concepts into interpretable propositions and applies logical composition to support reasoning, analysis, and control within LLM activations. Its modular pipeline combines concept dictionaries, activation thresholds, proposition matrices, and enriched representations for downstream intervention.

  • 3 ACTIVATIONREASONING: AR comprises latent representation discovery, proposition activation, and logical reasoning over detected concepts.Concept representations are collected in a dictionary, activations are mapped to propositional units, and logical rules compose higher-order structures.
  • 3.1 FINDING LATENT REPRESENTATIONS: Each concept is a tuple of a semantic identifier, latent representation, and soft activation threshold.The representation maps latent codes to nonnegative activation scores, and the resulting concept dictionary supplies building blocks for reasoning.
  • 3.1 FINDING LATENT REPRESENTATIONS: SAEs project entangled activations into sparse features that better support human-interpretable concepts and propositional reasoning.Concepts may be assigned manually from unambiguous features or induced automatically from labeled token data.
  • 3.1 FINDING LATENT REPRESENTATIONS: AR supports single-feature, multi-feature, and relational representations to balance interpretability with coverage of fragmented and context-dependent concepts.Relational representations use decision trees to express interactions such as conjunctions and context-dependent exclusions.
  • 3.2 ACTIVATING PROPOSITIONS: AR converts concept scores into local and global weighted evidence, allowing individual concepts to function as logical propositions.Local activations preserve token-level evidence, whereas global activations aggregate evidence across a sequence after concept-specific thresholding.
  • 3.3 LOGICAL REASONING: The enriched matrix A′ combines detected and inferred propositions, enabling failure tracing, compositionality analysis, safety evaluation, and actionable model control.This separates raw activation evidence from the structured representation produced after logical reasoning.

4 EXPERIMENTAL EVALUATIONS

AR is evaluated on multi-hop deduction, indirect concept descriptions, naturalized reasoning, and context-sensitive safety across two model backbones. Across these settings, it remains more robust to increasing complexity and linguistic or conceptual variability than the reported baselines.

  • PrOntoQA: PrOntoQA exposes a complexity advantage: larger instruction-tuned and reasoning models drop sharply as reasoning hops increase, whereas AR is reported to solve multi-hop tasks without degrading with complexity.The benchmark evaluates 1-, 3-, and 5-hop deductive questions.
  • 4 EXPERIMENTAL EVALUATIONS: AR consistently improves reasoning across PrOntoQA, Rail2Country, and ProverQA, remaining robust as hop count, abstraction, and linguistic difficulty increase.Table 1 covers exact-match accuracy for multi-hop, explicit and implicit concept descriptions, and diverse natural-language reasoning.
  • Rail2Country: On Rail2Country, AR reaches 75%/94% on explicit concepts and 86% on implicit concepts, while vanilla SAEs achieve 0% implicit-concept detection accuracy.AR (Gemma) outperforms larger or reasoning-model baselines in the reported comparisons.
  • ProverQA: On ProverQA, AR reaches 93%/91%/71% and 94%/91%/70% for easy/medium/hard tasks on Llama and Gemma, respectively.On hard problems, AR remains near 70% while all other reported models fall below 50%.
  • BeaverTails: On BeaverTails, AR improves context-sensitive safety classification over base SAEs, with hierarchical concepts achieving the strongest overall performance.Misinformation Regarding Ethics, Laws, and Safety and Non-Violent Unethical Behavior remain especially challenging; Table 2 identifies relational AR as best overall.

5 DISCUSSION

AR grounds propositions in SAE-derived features and improves reasoning across compositional, indirect-cue, linguistically diverse, and safety tasks. Its scope is bounded by imperfect concept features, unexplored large-scale settings, and fixed representation types.

  • AR maintains above 93% accuracy across 1–5 PrOntoQA hops while baselines degrade with complexity.It also surpasses larger instruction-tuned, proprietary, and reasoning-model baselines across the four experiments.
  • AR remains robust to indirect simile-based cues in Rail2Country, where small and large LLMs deteriorate and vanilla SAEs collapse.
  • AR retains strong performance on linguistically diverse ProverQA hard-tier problems, while all other models fall below 50%.
  • Current SAEs can be polysemous, context-dependent, or unable to capture abstract notions, although AR mitigates some effects with thresholding and richer representations.Pretrained SAEs also limit control over which concepts are discovered and how they are represented.
  • Large-scale open-ended, long-context, and knowledge-intensive reasoning remain unexplored, and AR currently fixes one representation type per concept.Automatic selection or hybrid representation types are proposed extensions.

6 CONCLUSION

AR integrates explicit logical reasoning into LLM latent activations, making opaque representations composable, steerable, and auditable. The framework is presented as a step toward more reliable reasoning and closer alignment with human values, while future work targets broader reasoning and representation methods.

  • AR grounds propositions in latent activations to support explicit composition, disambiguation, and inference over opaque representations.
  • AR turns latent activations into a substrate for structured reasoning, direct control and alignment interventions, and auditable safety mechanisms.
  • Future work includes automatic rule discovery, probabilistic or inductive reasoning modules, and automatic selection of concept representation types.
  • Extending AR beyond LLMs to vision or VLMs and evaluating large-scale open-ended, long-context tasks are identified as future directions.

A EXPERIMENTAL SETUP

The experiments evaluate AR on Llama and Gemma backbones using SAE-derived concept representations, proposition activation, symbolic rules, and steering. Setup details vary across PrOntoQA, Rail2Country, and ProverQA, with distinct datasets, thresholds, and model inputs.

  • AR configuration: AR represents concepts with SAE features, detects proposition activations, and applies logical rules; PrOntoQA and Rail2Country use multi-feature representations with mean aggregation and mean-shift steering.
  • PrOntoQA: PrOntoQA uses 500 samples for concept extraction and 2,000 unseen samples per hop subtask for binary True/False evaluation.
  • PrOntoQA: The PrOntoQA example encodes ontology statements as rules such as zumpus → ¬fast and derives facts for Alex from linked predicates.
  • Rail2Country: Rail2Country separates explicit color cues from meta-level descriptions, using 300 train, validation, and test samples for each setting.AR receives color codes through the logic component, whereas the baseline receives them in text.
  • ProverQA: ProverQA extracts propositions from dataset examples, adds 10 random sentences for counterexamples, and successfully identifies proposition activations without surrounding context.

A.4 SAFETY EXPERIMENTAL SETUP

The safety experiments use sentence-level concepts with three complementary constructions, and AR adds SAE processing plus forward logical reasoning to the base model. Runtime remains only marginally above the plain baseline while outperforming slower reasoning alternatives in accuracy.

  • Experimental setup: AR’s safety setup uses sentence-level concepts and evaluates tree-based, multi-feature SAE, and single-feature SAE constructions.All configurations aggregate activations with the mean operator and use mean-shift steering; safety experiments use Llama3.1 8B.
  • Tree-based concepts: Tree-based concepts retain up to 32 features per sentence and capture abstract relations by combining safety-relevant evidence.The decision trees are grown to depth 5.
  • SAE concepts: Multi-feature SAE concepts aggregate up to five of 32 retained sentence-level features, while single-feature concepts restrict each concept to one feature.Both settings allow multiple concepts to activate for the same sentence.
  • Runtime and accuracy: AR is only marginally slower than the plain baseline, substantially faster than chain-of-thought and reasoning models, and outperforms all alternatives in task accuracy.The implementation is not fully optimized, so further engineering may reduce the runtime gap.
  • Inference cost: AR requires an SAE forward pass and a logic-solver forward pass beyond the base model, adding modest runtime overhead.The added steps are described as lightweight compared with an LLM forward pass.

F RAIL2COUNTRY: EXTENDED RESULTS

Extended Rail2Country analyses show that AR improves detection and reasoning across explicit and simile-based concepts, with robustness across similes, countries, and adjacent layers. The remaining weakness is concentrated in more ambiguous visual associations.

  • Overall results: AR restores near-perfect detection and robust reasoning on both explicit R2C-Mono and implicit R2C-Meta concepts, whereas vanilla SAEs and base LLMs degrade sharply on similes.Vanilla SAE detection falls to 0% on R2C-Meta, while base LLM reasoning declines from 41/35% on R2C-Mono to 30/26% on R2C-Meta.
  • Per-simile breakdown: AR recovers most simile-based color cues at or near 100%, but yellow and white associations such as “fresh snow” or “banana” remain somewhat harder to ground.Without AR, simile detection is 0% across all cases.
  • Per-country breakdown: With AR, reasoning accuracy rises sharply and stabilizes across countries, and its degradation on implicit descriptions is relatively small compared with base-model collapse.Many countries are near zero without AR, while most reach close to 100% with AR.
  • Robustness: The gains remain consistent across languages, countries, and metaphorical variants, indicating generalization from explicit lexical mentions to abstract meta-level descriptions.The extended analyses show that aggregate improvements are not driven by only a subset of conditions.
  • Layer sensitivity: Adjacent-layer results remain highly consistent, with performance of 93.7 for layer 18 and 93.3 for layer 22.The ablation uses auto-thresholding, Rsingle concepts, and steering factors of 0.8 and 0.7, respectively.

H HYPERPARAMETER ABLATIONS

The PrOntoQA ablation finds stable AR performance across broad hyperparameter ranges, with moderate steering most sensitive and SAE-feature variability remaining an open stability challenge.

  • Hyperparameter robustness: AR remains stable across variations in multi-feature size, incoming-feature count, steering factor, and thresholding strategy on PrOntoQA with Gemma2 9B.The strongest results occur for Rmulti sizes between 7 and 9, top-kin = 2, and moderate steering around α = 0.4.
  • Sensitivity: The steering factor is the most sensitive component, while automatic thresholding performs on par with manually tuned values.These results indicate that reliable behavior does not require fragile or finely tuned configurations.
  • Open limitation: AR inherits variability from SAE-derived activations, which can fluctuate or remain partly polysemantic; improving SAE stability remains an open challenge.The authors identify advances in SAE activation stability as complementary and directly beneficial to AR.

J REPRODUCIBILITY STATEMENT

The paper supports reproducibility by providing implementation materials and using public benchmarks alongside the newly introduced Rail2Country dataset. It also states that language models assisted only with manuscript polishing, not scientific work.

  • Reproducibility resources: The authors provide framework, dataset, backbone, and hyperparameter details, plus code, preprocessing scripts, and rule sets for replication and extension.Public benchmarks are available, and Rail2Country is intended for release with the final version and included in the supplementary material.
  • LLM assistance: The authors report using LLMs only to polish and rephrase manuscript text, while retaining responsibility for ideas, methods, experiments, analysis, and conclusions.The stated use was limited to improving readability.
Loading 2510.18184v3…