PaperlayerView exact revision on arXiv

Source-linked AI summary

Language Models are Injective and Hence Invertible

Giorgos Nikolaou, Tommaso Mencattini, Donato Crisostomi, Andrea Santilli, Yannis Panagakis, Emanuele Rodolà

arXiv:2510.15511v4cs.LGcs.AI

TL;DR

The paper addresses whether Transformer representations lose input information because of nonlinearities, normalization, and many-to-one attention. It proves almost-sure injectivity for standard decoder-only Transformers and introduces SIPIT for exact recovery from hidden activations. The authors conclude that injectivity is a rigorous, operational property with implications for interpretability, auditing, and privacy.

  • Problem

    The paper examines whether nonlinearities, normalization, and many-to-one attention cause distinct inputs to collapse into identical Transformer representations, preventing exact recovery.

  • Method

    The authors mathematically analyze real-analytic Transformer mappings and gradient-descent training, then introduce SIPIT to reconstruct prompts from hidden activations.

  • Results

    Standard decoder-only Transformers are almost surely injective, and SIPIT recovers exact input sequences with provable linear-time guarantees.

  • Takeaways & Limitations

    Hidden states should be treated as information-preserving representations of prompts, providing a baseline for interpretability and auditing.

  • Takeaways & Limitations

    The analysis does not yet establish how invertibility extends to multimodal models or remains robust under noise and quantization.

Abstract

from arXiv · show

Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exact recovery of the input from a model's representations. In this paper, we challenge this view. First, we prove mathematically that transformer language models mapping discrete input sequences to their corresponding sequence of continuous representations are injective and therefore lossless, a property established at initialization and preserved during training. Second, we confirm this result empirically through billions of collision tests on six state-of-the-art language models, and observe no collisions. Third, we operationalize injectivity: we introduce SipIt, the first algorithm that provably and efficiently reconstructs the exact input text from hidden activations, establishing linear-time guarantees and demonstrating exact invertibility in practice. Overall, our work establishes injectivity as a fundamental and exploitable property of language models, with direct implications for transparency, interpretability, and safe deployment.

1 INTRODUCTION

The paper argues that standard decoder-only Transformers are almost surely injective: distinct prompts produce distinct hidden representations despite nonlinearities and normalization. It combines a mathematical proof with SIPIT, an algorithm for exact reconstruction from activations.

  • Main result: Standard decoder-only Transformers are almost surely injective, so distinct prompts yield distinct last-token representations across essentially all parameter settings and training stages.The guarantee applies under common initializations and finite training horizons, excluding only measure-zero pathological parameters.
  • Operationalization: SIPIT uses gradient-based reconstruction to recover prompts exactly from hidden activations with provable linear-time guarantees, turning injectivity into an operational tool.The algorithm is presented as a practical consequence of the theoretical guarantee.
  • Proof strategy: The proof treats Transformers as functions whose real-analytic components make collisions a measure-zero event, while gradient descent preserves injectivity after any fixed number of steps under mild assumptions.The argument constructs collision-free parameter settings and shows that training does not move parameters into the exceptional set.
  • Conceptual contribution: The paper reframes injectivity as a structural consequence of the architecture rather than an asymptotic idealization, establishing that last-token states almost everywhere identify the input.The analysis attributes possible collisions to deliberate non-analytic choices such as quantization or parameter tying.

2 TRANSFORMERS ARE INJECTIVE

Decoder-only Transformers almost surely map distinct finite prompts to distinct last-token hidden states. Real-analytic structure, continuous initialization, and gradient-based training together preserve this injectivity, except for deliberately engineered parameter choices.

  • Main result: Almost surely, decoder-only Transformers map every pair of distinct prompts to different last-token hidden states.The guarantee holds for finite vocabulary and context length under random initialization with a density and finite gradient-descent training.
  • Proof strategy: Real-analytic architecture components confine prompt collisions to measure-zero parameter sets rather than ordinary operating conditions.The argument covers embeddings, positional encodings, normalization, attention, analytic MLP activations, and residual connections.
  • Training preservation: A single gradient-descent update preserves absolute continuity because its analytic update map is locally invertible away from a measure-zero Jacobian-singularity set.Applying this argument at every step prevents training from concentrating parameters on collision sets.
  • Training preservation: The injectivity guarantee also covers SGD, mini-batch, and full-batch updates, and extends from pairwise prompts to every finite prompt set.The stated corollaries allow arbitrary, including random or adversarial, batch selections under the theorem’s assumptions.
  • Failure cases: Deliberately duplicated token or positional embeddings can manufacture collisions, although such engineered choices are excluded almost surely by continuous initialization and standard training.Quantization and other deliberate non-analytic choices are also identified as potential collision mechanisms.

3 EXACT PROMPT RECOVERY VIA SIPIT

SIPIT converts Transformer injectivity into sequential exact prompt recovery from hidden activations. It tests candidate tokens against the observed state at each position and has a probability-one bound of T|V| trials.

  • Scope: SIPIT’s threat model assumes access to a hidden-state sequence, with examples including leaked KV-caches, shared inference, or APIs exposing intermediate representations.The paper distinguishes this setting from defining a complete adversarial model.
  • Sequential recovery: Given a recovered prefix, causal structure makes each position’s hidden state identify the unique next token.The hidden state depends only on the preceding prefix and current token, enabling sequential reconstruction.
  • Algorithm: At each position, SIPIT cycles through vocabulary candidates, verifies a predicted-state match within tolerance, appends the match, and proceeds.The algorithm initializes an empty sequence, tracks tested candidates, and returns the reconstructed sequence after processing all positions.
  • Correctness and complexity: SIPIT recovers the true input sequence with probability one in at most T|V| candidate-token trials.The guarantee follows because local injectivity gives one matching token at each position and vocabulary search eventually tests it.
  • Implication: The method is presented as an operational consequence of structural injectivity, providing linear-time exact recovery and often running faster in practice.This connects the uniqueness theorem to a constructive reconstruction procedure rather than treating injectivity as a purely theoretical property.

4 EXPERIMENTS

The experiments find no representation collisions across models, layers, prompt lengths, and quantization settings, while SIPIT achieves exact prompt recovery efficiently.

  • 4.1 SEARCHING FOR COLLISIONS: Around 5 billion pairwise comparisons across 100k prompts found no collisions across all tested models and layers.Minimum distances remained strictly positive in the reported model families and layers.
  • 4.1 SEARCHING FOR COLLISIONS: Minimum, mean, and maximum pairwise distances rise at short sequence lengths and then stabilize, indicating persistent separability as prompts grow.The bounded spread showed no sign of pathological collapses in GPT-2 Small, with similar behavior reported for Gemma3.
  • 4.1 SEARCHING FOR COLLISIONS: More than 343 billion continuation-prompt pairs per model also produced distances far from zero, confirming local injectivity in an exhaustive test.The test focused on prompts with the smallest observed last-token distances and appended every vocabulary token.
  • 4.1 SEARCHING FOR COLLISIONS: Quantization introduced no collisions across tested models and more than doubled minimum representation distances.The experiments covered FP4 and INT8 settings, including larger quantized models.
  • 4.2 INVERTIBILITY RESULTS: Under FP4 weight quantization, SIPIT recovered all inputs perfectly while exploring less than 0.22% of the vocabulary on average.This robustness test used models with approximately 32K and 128K vocabularies.
  • 4.2 INVERTIBILITY RESULTS: 100% token-level accuracy: SIPIT exactly recovers every tested 20-token prompt, whereas HARDPROMPTS fails and BRUTEFORCE requires orders-of-magnitude longer runtimes.The comparison evaluates training-free inversion from hidden states.

5 RELATED WORK

The paper distinguishes its discrete-to-continuous injectivity analysis from prior work on non-injective components and approximate or black-box prompt inversion. It presents SIPIT as training-free, efficient, and exactly recoverable with provable linear-time guarantees.

  • Analytical properties of Transformers: The paper studies the discrete-to-continuous map from prompts to hidden states, rather than treating individual Transformer components as standalone maps.This framing supports finite-width, finite-depth, finite-horizon guarantees under analyticity assumptions.
  • Inverse problems in language modeling: Prior inversion methods commonly use generated outputs, logits, or auxiliary inverters and generally produce approximate reconstructions at high computational cost.These methods are described as complementary rather than directly comparable to SIPIT's hidden-state setting.
  • Inverse problems in language modeling: Unlike prior approaches, SIPIT performs training-free, efficient, exact recovery from internal states with provable linear-time guarantees.The paper positions this setting as distinct from black-box methods and approximate prompt optimization.

6 DISCUSSION AND CONCLUSIONS

The paper concludes that decoder-only Transformers are almost surely injective and that SIPIT turns this structural property into exact inversion. It argues that hidden states can therefore carry direct transparency, interpretability, privacy, and deployment consequences, while broader extensions remain open.

  • 6 DISCUSSION AND CONCLUSIONS: Decoder-only Transformers are almost surely injective, and SIPIT recovers exact input sequences with provable linear-time guarantees.Together, these results make injectivity an operational property rather than an informal assumption.
  • 6 DISCUSSION AND CONCLUSIONS: Injectivity is presented as a foundation for causal and probing analyses because last-token states faithfully encode the full input.This consequence is stated specifically for mechanistic interpretability.
  • 6 DISCUSSION AND CONCLUSIONS: Because hidden states can be recovered as the user's exact text, systems that store, cache, or transmit them may face the same data-protection obligations as raw prompts.The paper distinguishes inference-time hidden representations from training data encoded in model weights.
  • 6 DISCUSSION AND CONCLUSIONS: The paper identifies multimodal extensions, approximate inversion under noise or quantization, and regulatory integration as open directions.These directions define the stated boundary of the current work.

REPRODUCIBILITY STATEMENT

The reproducibility materials define the model, sequence, parameter, matrix, and real-analytic notation used throughout the paper. They also document the proofs, implementation details, and formal assumptions underlying the analysis.

  • Reproducibility materials: The paper records complete proofs and implementation details in the appendices, including analyticity verification for activation functions used in modern LLMs.The experimental setup is documented separately in Section 4, including hardware, software versions, and dataset construction.
  • Notation and model definition: The paper defines the model as f: V≤K × R^p → R^d over a finite vocabulary and bounded context length, with outputs representing final token embeddings.The finite input domain reflects practical context limits.
  • Real-analytic preliminaries: The appendix states the zero-set principle that a nontrivial real-analytic map has a measure-zero zero set, supporting the collision arguments.The vector-valued extension follows by containment in the zero set of a nontrivial scalar component.
  • Real-analytic preliminaries: Real-analyticity is extended from vector-valued functions to matrix spaces using componentwise definitions and vectorization or matricization.Composition and closure properties support the later Transformer analyticity arguments.

B TRANSFORMER LANGUAGE MODEL

This section specifies the decoder-only Transformer as a composition of embeddings, causal attention, normalization, MLPs, residual blocks, and output layers. It also establishes that these components are jointly real-analytic in inputs and parameters.

  • Input processing: The model maps token sequences into continuous representations through learned token and positional embeddings.Token embeddings select rows from a trainable matrix, while positional embeddings provide position-dependent vectors that are added elementwise.
  • Transformer backbone: Transformer blocks combine normalized multi-head self-attention and row-wise MLPs with residual connections.The blocks are stacked sequentially, with each block preserving the model width needed for residual additions.
  • Attention: Causal self-attention restricts each position to preceding and current positions while using trainable query, key, and value projections.The section gives equivalent masked and projection-form definitions of causal attention.
  • Full architecture: The complete language model composes the embedding layer, a stack of Transformer blocks, and an unembedding layer.The formal specification uses a finite vocabulary, finite context length, and parameter vector θ.
  • Analyticity: Under real-analytic activation assumptions, the embedding layer and core Transformer modules are jointly real-analytic in their inputs and parameters.This regularity underlies the later measure-zero arguments about representation collisions.

C ALMOST SURE INJECTIVITY

This section proves that causal decoder-only Transformers almost surely map distinct input sequences to distinct last-token representations. The guarantee holds after finite gradient-descent training under the stated architectural and initialization assumptions.

  • Assumptions: The result applies under standard architectural conditions, including embedding dimension d ≥4 and at least one attention head per block.The paper states that these assumptions are satisfied by modern large language models.
  • Main theorem: With probability one, the last-token representation map is injective after any finite number of gradient-descent steps.The theorem assumes finite vocabulary and context, at least one attention head per block, suitable widths, absolutely continuous initialization, and step sizes in (0, 1).
  • Training preservation: The proof preserves injectivity during training by showing that finite gradient-descent updates preserve absolute continuity of the parameter distribution.The update map and its Jacobian are analyzed using real-analyticity and measure-zero critical sets.
  • Almost-sure distinctness: Collisions are confined to measure-zero parameter choices, so absolutely continuous parameter laws yield pairwise distinct last-token representations almost surely.The argument first establishes distinctness for fixed pairs and then extends it to finite input families.

C.1 ABSOLUTE CONTINUITY ENSURES ALMOST SURE INJECTIVITY

This section establishes almost-sure injectivity by proving that distinct sequences admit parameter settings with distinct representations, then applying real-analytic zero-set arguments and a finite union bound.

  • Pairwise collisions: For any fixed pair of distinct inputs, the collision set has measure zero under the real-analytic representation map.A nonzero witness makes the collision function non-identically zero, allowing the zero-set theorem to apply.
  • Witness construction: Explicit witness constructions separate any two distinct sequences, whether they differ in length, final token, or an earlier token.Earlier differences are detected by orthogonal embeddings and attention, while length or final-token differences can be isolated through embeddings and positional encodings.
  • Finite families: Almost-sure pairwise distinctness extends to every finite input family by a union bound over the finitely many input pairs.Thus last-token representations are pairwise distinct almost surely across all inputs in the finite family.
  • Training distribution: Gradient descent preserves the absolute continuity needed by the injectivity argument through regularity of the update map and measure-zero critical sets.The proof uses real-analyticity, a nondegenerate witness, local diffeomorphic charts, and change of variables.

C.2.2 GRADIENT DESCENT PRESERVES ABSOLUTE CONTINUITY

This section shows that gradient descent preserves absolute continuity and connects that result to sequential inversion. Causality lets SIPIT recover tokens one position at a time with worst-case linear-time dependence on sequence length.

  • Absolute-continuity preservation: Gradient descent preserves absolute continuity after one step and after any finite composition of steps with η ∈ (0, 1).The result follows by excluding a measure-zero critical set, covering the regular region with countably many diffeomorphic charts, and applying change of variables.
  • Sequential inversion: Causality makes each hidden state depend only on the prefix through the current token, enabling token-wise inversion.The inversion procedure conditions on the recovered prefix and tests candidate tokens against their predicted hidden representations.
  • SIPIT: SIPIT achieves exact and robust recovery with worst-case time Θ(T|V|).It performs one pass over the vocabulary at each sequence position using a small-ball verifier.

D.1 ONE-STEP LAST-TOKEN MAPS

The one-step map from a candidate token under a fixed prefix to its hidden state is almost surely injective and positively separated. This local identifiability enables SIPIT to recover every token exactly, with at most |V| tests per position and Θ(T|V|) total verifier tests.

  • One-step injectivity: For a fixed prefix and position, the one-step map F from candidate tokens to hidden states is almost surely injective, so distinct tokens produce distinct representations.The proof applies the earlier injectivity result to the finite set of prefix-plus-token sequences.
  • One-step injectivity: The finite set of token representations has a strictly positive minimum pairwise distance, providing a separation margin for verification.Any finite collection of distinct points in R^d has a positive minimum pairwise distance.
  • Local verification: With exact observations, or noise below half the separation margin, the local verifier uniquely accepts the true token.The noiseless case uses ε = 0; robust uniqueness holds when ε < Δ_π,t/2.
  • Global inversion: Because candidate policies enumerate the finite vocabulary without replacement, every token is proposed once and the true token is accepted within |V| iterations.The enumeration is a permutation of V, so the true token must eventually be tested.
  • Global inversion: SIPIT therefore reconstructs the exact sequence almost surely, including under its stated uniform noise condition.The guarantee applies position by position and terminates after the outer loop has recovered all tokens.
  • Complexity: The verifier-test count is at most T|V|, giving a linear iteration bound in sequence length times vocabulary size.Wall-clock time additionally depends on each test and policy overhead, so it need not equal the iteration bound.

E.1 IMPLEMENTATION DETAILS

The experiments distinguish exact functional collisions from numerical near-collisions and test the latter with floating-point tolerances. SIPIT uses gradient clipping and periodic projection, while the comparison method is adapted to the text-only setting.

  • Collision criterion: Theoretical collisions mean exact equality of last-token representations for two distinct prompts, whereas experiments use a floating-point allclose criterion as a numerical proxy.The empirical criterion uses PyTorch’s relative and absolute tolerances of 10^-5 and 10^-8.
  • Collision criterion: Across billions to trillions of pairwise checks, no distinct prompt pair satisfied the numerical collision criterion.Observed ℓ2 distances remained bounded away from zero at the tested numerical precision.
  • SIPIT implementation: SIPIT implementation clips gradients at norm 1 and projects the continuous proxy to the nearest token embedding every K = 50 proposals.The implementation states that these heuristics affect efficiency but not the verifier or correctness guarantees.
  • Baseline implementation: HARDPROMPTS is adapted from multimodal models by replacing its CLIP-based objective with SIPIT’s ℓ2 objective and using T = 1 optimization step.Other step sizes and stopping rules mirror the SIPIT setup for comparison.

E.2.1 COLLISION EXPERIMENTS

Collision experiments find clear separation across model families, depths, sequence lengths, and deliberately matched next-token outputs. The closest qualitative pairs are near-duplicate sequences, while SIPIT is evaluated on in-distribution, unseen, and random inputs.

  • Model depth and scale: Across GPT-2 and Gemma-3 families, minimum pairwise distances remain orders of magnitude above 10^-6 at every depth, with separation generally increasing in deeper or larger models.Figure 7 covers GPT-2 Small, Medium, and Large; Figure 8 covers Gemma3-1B, 4B, and 12B.
  • Sequence length: For Gemma-1B, minimum, mean, and maximum pairwise distances rise quickly with short sequence lengths and then plateau without approaching zero.The observed pattern indicates stabilized rather than collapsing margins over the explored lengths.
  • Matched outputs: Across the reported collision and ablation experiments, last-token states remain well separated, including for prompts with identical next-token answers.The separator-token study compares ->, :, =, and - across translation and math prompt pairs.
  • Robustness evaluation: The experiments include train-distribution, Wikipedia test, and random-token prompts, each containing 50 sequences of length 100.These datasets are used to assess robustness on in-distribution, unseen, and out-of-distribution inputs.
  • Nearest neighbors: Closest qualitative pairs are near-identical code or documentation snippets differing mainly in trailing whitespace, yet their last-token representations remain clearly separated.A Llama pair has distance 1.274 and a Mistral pair has distance 1.146 at the final layer.

E.5 RELATION WITH ANISOTROPY AND INTRINSIC DIMENSION

The section relates injectivity to representation geometry, finding that anisotropy and intrinsic dimension correlate with numerical separation rather than information-destroying collapse. It also positions these observations as complementary to geometric analyses of representation evolution.

  • Experiment 2: intrinsic dimension vs. injectivity margin: Higher anisotropy and lower intrinsic dimensionality tend to coincide with larger injectivity margins, meaning continuation representations are more separated.The paper describes compressed-looking manifolds as, if anything, more separated.
  • Discussion: The analysis complements the injectivity framework by diagnosing how representations evolve geometrically across depth and training.The injectivity results establish almost-sure losslessness, while the related analysis provides fine-grained geometric diagnostics.
  • Discussion: The preliminary depth-wise picture combines growing margins, decreasing intrinsic dimension, and a concave anisotropy trajectory with a late spike.The section presents this as an alternative to learning explanations based purely on layer-by-layer bottleneck compression.

F REAL-ANALYTIC ACTIVATION FUNCTIONS IN MODERN LLMS

The section checks whether the real-analytic activation assumption used by the injectivity theory matches modern language-model practice. It finds that the surveyed architectures use analytic activations and explains how these functions support the theoretical regime.

  • Empirical survey: All 18 surveyed models use real-analytic feed-forward activations, supporting the relevance of the injectivity results to modern language models.The surveyed activations include SiLU/Swish, SwiGLU, GeGLU, and GELU.
  • Analyticity of common activations: SiLU and GELU are real-analytic scalar functions, and their coordinatewise vector extensions remain real-analytic.The paper derives the scalar properties from analytic elementary functions and then lifts them componentwise.
  • Analyticity of modern blocks: GLU-style feed-forward blocks preserve real-analyticity because they combine affine maps, analytic activations, composition, and coordinatewise products.The proof explicitly uses closure of real-analytic functions under these operations.
  • Relation to expressivity theory: The theory concerns typical decoder-only training behavior under real-analytic activations, rather than claiming that every Transformer function must be injective.Non-injective Transformers can exist, but the non-injective parameter set has measure zero and standard training almost surely avoids it.
Loading 2510.15511v4…