PaperlayerView exact revision on arXiv

Source-linked AI summary

Semantic Fibers and Cross-Gram Interference: A Calculus of Safety Drift in Overcomplete Representations

Mohammed Ahnouch, Lotfi Elaachack

arXiv:2609.14861v1cs.LGcs.AI

TL;DR

Cross-lingual refusal failures motivate a geometric audit of safety-margin drift across externally audited same-intent prompts. The paper characterizes drift through representation and read/write geometry, then uses intrinsic exposure and conditioning to distinguish repairability regimes. Its conclusion is that similar observed exposure can imply different remediation, while the evidence is bounded by the settings studied.

  • Problem

    A model can refuse a harmful request in English yet comply with its faithful translation, and behavior or direction transfer alone leaves the geometric diagnosis open.

    Source: §1 · p. 2

  • Method

    The paper fixes an audited equivalence relation and local representation, characterizes same-fiber drift through cross-Gram geometry, and uses intrinsic exposure and conditioning to diagnose repairability.

    Source: §1 · pp. 2–3

  • Results

    The audit distinguishes reader faults, unstable exact corrections, and representation-level collisions, which can require different remedies despite similar observed exposure.

    Source: §1 · p. 3, §8 · p. 33

  • Takeaways & Limitations

    High current exposure with low intrinsic exposure points to reader-side recalibration or projection, whereas high intrinsic exposure shifts repair to the representation.

    Source: §8 · p. 33

    The trained-frame findings use toy reconstruction networks under one attack, so their vulnerability claim is limited to that setting rather than language models.

    Source: §6.2 · p. 30

Abstract

from arXiv · show

A deployed language model may refuse a harmful request in English yet comply with its faithful translation, revealing a cross-lingual safety failure that cannot be characterized reliably by output behavior alone. We formalize this phenomenon through an audited equivalence relation and show that, for a declared quotient, representation, metric, feature dictionary, scoring head, threshold, and contrast model, the resulting safety drift admits an exact linear-algebraic characterization. Specifically, the drift is a cross-Gram functional of the within-fiber contrast; its worst admissible value is a support function, while margin invariance is characterized by an annihilator condition. We introduce an intrinsic calibrated exposure measure, governed by the leverage duality $\chi^2=1/\ell-1$, which separates observed drift into three diagnostically distinct regimes: a reader fault removable by recalibration, an exact correction that is too ill-conditioned to be reliable, and a representation-level collision that no readout-only intervention can remove. Thus, identical observed exposure can lead to fundamentally different remediation verdicts. The framework also extends to cone-valued safety heads. An untied order-swap identity provides a diagnostic for the linear control interface; its calibration-state residual predicts a distinct three-control composition error on unseen states and targets, achieving median Spearman correlation $0.964$, compared with $0.269$ for a static cross-Gram baseline. etc.....

1 Introduction

A refusal that holds in English but fails on a faithful translation motivates an audit of same-intent drift, fixed by an externally declared equivalence relation. The paper characterizes that drift geometrically and uses intrinsic exposure and conditioning to distinguish three repair regimes.

1 Introduction

  • High current exposure with low intrinsic exposure is a reader fault; low intrinsic exposure with a large invariant-reader norm makes exact correction unstable; high intrinsic exposure rules out a readout-only fix.The corresponding work is recalibration or projection, regularized reading, or representation-level repair, respectively.Source: §1 · p. 3
  • Same-fiber safety-margin drift is a cross-Gram functional of the contrast, and margin invariance holds exactly under an annihilator condition.Worst admissible drift is a support function, with the governing quantity depending on the declared budget.Source: §1 · p. 3
  • Intrinsic calibrated exposure obeys χ_B^2 = 1/ℓ_B − 1, while the invariant-reader norm 1/sin θ_s,B separates feasible cancellation from stable, usable cancellation.Identical observed exposure can therefore call for opposite remediation.Source: §1 · pp. 3–4
  • An untied order-swap identity tests whether a steering stack follows the assumed linear read/write interface, while two head-only repairs offer regularized reading and least-change projection.A column permutation can move block exposure without changing the Gram spectrum, and optimized cross-talk has a rank floor attained by equal-leverage frames.Source: §1 · p. 4
  • The running audit computes exposure from paired activations and routes low-exposure incidents toward reader repair, while high intrinsic exposure shifts the repair budget to the representation.Applying the calculus to a model requires an audited fiber set and the declared layer, dictionary, head, and equivalence relation.Source: §1 · p. 4

2 Fibers, writers, and readers

The framework treats a fiber as an externally audited set of prompts whose policy-relevant intent is held fixed, then models their local activations through feature writers and readers. Its declared representation, metric, calibration, and contrast model determine the drift quantities and the scope of any certificate.

2 Fibers, writers, and readers

  • A fiber collects audited translations, paraphrases, or other policy-equivalent inputs, while its realized contrasts and their linearized span distinguish observed changes from the subspace used for radius statements.A ball in that span can contain directions no prompt realizes, so prompt-level certification requires realized contrasts or an audited feasible set.Source: §2 · p. 5
  • At a fixed layer, the local model writes hidden activations as h(x) = Wz(x) + ε(x), with W supplying writer directions and ε bounded by η.The columns are unit-normalized unless stated otherwise.Source: §2 · p. 5
  • Readers measure feature activity and need not coincide with writers; self-calibration fixes each reader's response to its own writer, and their cross-Gram matrix records directional read/write cross-talk.In the tied case R = W with unit columns, the cross-Gram matrix reduces to the ordinary Gram matrix.Source: §2 · pp. 5–6
  • The safety head is a reader with margin M(x) = u⊤h(x) − τ, calibrated against a designated safety writer; its pullback block row is the cross-Gram segment when the head is a reader column.Under the stated calibration, intrinsic exposure is no greater than current exposure.Source: §2 · p. 6
  • In the implementation mapping, h is a fixed-layer activation, W can be a sparse-autoencoder dictionary, and u can be a linear refusal probe; contrasts and radii use the declared metric and normalization.The benchmark estimates readers on training fibers, selects on validation fibers, and evaluates on untouched test fibers.Source: §2 · pp. 6–8

3 Exposure and certificates

For a declared representation and audited fiber, same-fiber margin drift is a linear functional of the within-fiber contrast, and margin invariance is an annihilator condition. Support functions turn that geometry into decision certificates, while the measured results show that exposure depends on the chosen budget and drift distribution.

3.1 Drift and sensitivity constants

  • With zero residual, same-fiber margin drift is v applied to the feature contrast, or the cross-Gram row applied to a block contrast; the margin is invariant exactly when v annihilates the contrast span.This requires annihilating within-fiber contrasts, not collapsing the hidden states.Source: §3.1 · pp. 8–9
  • The feature-budget constant ρ measures worst-case drift per unit feature-code norm, κ measures it per unit hidden-state distance, and σ∆ gives the stochastic drift scale.These quantities are not interchangeable, and decision failure also depends on the baseline-margin distribution.Source: §3.1 · pp. 9–10

3.2 Decision certificates and measured exposure

  • For an admissible contrast set, the support function of the negative drift direction certifies whether every contrast preserves a positive margin or some contrast can make it nonpositive.For a compact set, equality with the baseline margin is first contact with the decision boundary.Source: §3.2 · p. 10
  • The directed unsafe rates are 0.000%, 1.472%, and 30.721% for protected, random, and spiked families; under a hidden-distance budget, drift distribution, not ρ, tracks the separation.At fixed hidden radius, κ reaches 0.670/1.000/1.000 while ρ stays constant by construction.Source: §3.2 · p. 12, Figure 3 · p. 13
  • The Gaussian prediction was 25.91% for the spiked family versus 30.721% observed, with varying sparse supports producing a heavier-tailed drift distribution.The fixed-σ∆ prediction is described as a floor on the directed rate.Source: §3.2 · p. 12
  • At n/d = 3.00, feature-budget norms are 0.000, 0.899, and 3.464 for protected, random, and spiked families; hidden-budget norms are 0.000, 0.876, and 0.991.The feature-budget spiked/random ratio is about 3.9, versus about 1.13 for the hidden-budget norms.Source: §3.2 · p. 12, Figure 3 · p. 13
  • 31.4% of fibers were directed unsafe failures and 32.7% were reverse-directed; the sign boundary identifies failures as points with ∆M ≤ −Men.The two directions are near-symmetric as the construction requires.Source: §3.2 · p. 13, Figure 4 · p. 14

4 Diagnosis

The diagnosis separates removable reader cross-talk from representation-intrinsic exposure and from exact correction that is too ill-conditioned to be reliable. Matched-exposure tests show that observed exposure alone does not determine repairability.

4.1 Intrinsic exposure, leverage duality and conditioning

  • The theorem separates removable readout cross-talk from residual χB, which is representation-intrinsic under the declared calibration constraint.The scalar-writer formulation also extends to k calibrated outputs, where simultaneous invariance is feasible exactly when the projected cone has rank k.Source: §4.1 · pp. 15–16
  • When endpoint reconstruction error is bounded by η, exact cancellation can remain brittle because the invariant reader amplifies residual noise and estimation error by 1 / sin θs,B.This conditioning regime is not captured by current exposure or intrinsic exposure alone.Source: §4.1 · p. 16
  • The scalar-writer model is contested because refusal may involve multiple directions or concept cones, and orthogonality does not imply intervention independence.The paper gives a multi-output extension for cone-valued safety heads.Source: §4.1 · p. 16

4.2 The three regimes, matched and measured

  • The angle from the safety writer to the nuisance span distinguishes stable removal, ill-conditioned removal, and intrinsic collision, where no invariant calibrated reader exists.A right angle is stable, a small positive angle is removable but ill-conditioned, and an in-span writer is an intrinsic collision.Source: §4.2 · p. 16
  • At matched ρ = 0.880, validation-selected repair reduces held-out drift by 71.0% in the stable-removable family, 12.7% in the ill-conditioned-removable family, and −0.0% in the intrinsic family.The families have intrinsic exposure χ = 0.000/0.000/0.880, despite identical current exposure.Source: §4.2 · p. 18, Figure 6 · p. 19
  • Across 66 independently rotated geometries, χ/ρ predicts residual drift floor (Spearman 0.972), while on removable cases reader norm predicts usable repair (Spearman 0.889).The two quantities distinguish irreducible drift from repair limited by the utility constraint.Source: §4.2 · p. 19, Figure 7 · p. 20

5 The causal interface

The order-swap residual tests whether controls follow the linear reader/writer interface and predicts three-control composition error on unseen states and targets. The paper derives the diagnostic for untied and tied systems, then checks it on synthetic interventions.

5 The causal interface

  • Untied setters commute for all states and targets exactly when both directional cross-Gram entries, Cij and Cji, are zero.The order-swap identity gives the commutation defect from the two directional reader-writer interactions.Source: §5 · pp. 20–21
  • The observed-minus-predicted residual localizes nonlinear leakage, state dependence, off-dictionary components, or intervention implementation error.Its diagnostic value is in the residual rather than the predicted linear term.Source: §5 · p. 21
  • For tied readers and writers, the defect is governed by feature coherence gij; all setters commute exactly when gij is zero.Independent centered targets with variance σ^2 yield expected squared defect 2σ^2gij^2.Source: §5 · p. 22
  • Tied synthetic interventions recover the block norm as 1.939 against the true 1.933, but these recoveries check identity consistency rather than provide independent causal evidence.The section presents this as a template for estimating exposure on a real stack where W is unavailable.Source: §5 · p. 22
  • 0.964 median within-system Spearman correlation (0.970 pooled over 360 held-out triples), versus 0.269 (0.298 pooled) for static cross-Gram magnitude alone.The calibration-state residual predicts unexplained three-control composition error on new states and targets.Source: Figure 8 · p. 23, §5 · p. 23

6 Repair and limits

The section distinguishes reader-side repairs from representation-level limits, then shows that aggregate frame statistics can conceal block-specific safety exposure. Its experiments show that the usefulness of aggregates depends on how feature importance is allocated.

6 Repair and limits

  • The framework contrasts learned-classifier erasure with a calibrated, block-specific verdict: it designs a reader, edits the deployed head with a held-out bound, or identifies when neither can work.When χB > 0, head-side repair cannot remove the block response.Source: §6 · p. 24

6.1 Two closed-form repairs

  • As λ increases, the regularized reader approaches the normalized tied reader; as λ decreases, it approaches the minimum-exposure reader, tracing a frontier between deployed behavior and nuisance cancellation.The section treats λ as a design knob and reports one frozen test evaluation, not a test-set optimum.Source: §6.1 · pp. 24, 26
  • For multi-output safety heads, exact simultaneous invariance is feasible precisely when the projected anchor matrix has full column rank.This permits multiple policy dimensions without collapsing them into one scalar direction.Source: §6.1 · p. 25
  • The regularized reader designs a calibrated head from block geometry or contrast covariance, while the least-change projection uses audited pairs and changes the deployed head minimally.When χB > 0, neither repair removes the block response, so the diagnosis points to representation-level work.Source: §6.1 · p. 26
  • On untouched test fibers, mean absolute drift fell from 1.394 to 0.000 with oracle projection and 0.008 with data PCA truncation; directed unsafe rate fell from 28.79% to 0.04% and 2.16%.At τ = 0.65, the tied rate was 33.15% against 0.00% for both projections.Source: §6.1 · p. 26, Figure 9 · p. 27

6.2 What aggregates fix, and what they leave free

  • Equal overcompleteness and global coherence can coexist with any block exposure, while equal-leverage frames attain the reader-optimized floor and spread cross-talk evenly.A column permutation preserves spectrum, rank, trace, and frame potential while changing block exposure.Source: §6.2 · pp. 27–28
  • At the capacity limit, equal-leverage frames attain the reader-optimized floor at ratio 1.000; random unit frames average 1.022 of the floor, while tied readers retain 1.552 times optimized cross-talk.The gain from reader optimization tracks leverage unevenness (Spearman 0.752).Source: §6.2 · pp. 28–29
  • Uniform-importance frames track attacked loss (R2 = 0.82), but with decaying importance neither level (R2 = 0.11) nor global worst-case interference (R2 = 0.19) does.Decaying importance protects the most important feature to 0.248 of median-feature interference; these toy reconstruction networks use one attack, so the finding is not about language models.Source: §6.2 · pp. 29–30

7 The audit in practice

The audit proceeds from declared semantic fibers and a fixed score through exposure and repairability measurements to validation-selected repair. On one multilingual encoder, the regularized reader reduced test drift on audited and unaudited languages while retaining English-score correlation; the exact invariant reader was too ill-conditioned to use.

7 The audit in practice

  • Fibers are declared and split by semantic intent before activation analysis, then the score, operating point, contrast geometry, and exposure budget are fixed and reported.Validation selects regularization under a prespecified utility floor; the reader, threshold, and regularizer are frozen before the single test evaluation.Source: §7 · pp. 30–31
  • A layer-resolved profile identifies depths where intrinsic exposure is near zero and conditioning is moderate, indicating where a head-side repair may be available.The fibers are audited externally rather than inferred from feature overlap; the premise that same-meaning inputs share dictionary features across languages and scripts is measurable [Karne, 2026].Source: §7 · p. 31
  • Without a dictionary, the regularized reader is computed by a ridge-regularized contrast-covariance solve, with low-rank or iterative methods available for large hidden dimensions.The solution is renormalized to satisfy the calibration constraint exactly.Source: §7 · p. 31

7.1 The audit on a real encoder

  • The real-encoder pilot used a 384-dimensional mean-pooled final layer, 300 professionally translated FLORES-200 sentences, four audited languages, and three held-back languages.Sentences were split by fiber, so realizations of one meaning did not cross splits.Source: §7.1 · p. 32
  • The exact invariant reader had norm 4.9 and raised mean absolute test drift by −47%, while English-score correlation fell to 0.26.The 221-dimensional annihilator was estimated from training contrasts; the authors identify fitted noise as the reason the exact fix is unusable.Source: §7.1 · p. 32
  • On untouched fibers, regularization reduced mean absolute drift from 0.122 to 0.114 on audited languages and from 0.213 to 0.186 on unaudited languages, with English-score correlation at 0.92.Figure 11 compares the deployed direction, selected reader, and exact projection by language.Source: §7.1 · p. 32, Figure 11 · p. 33

8 Conclusion

The conclusion maps observed exposure and intrinsic exposure to distinct repair actions, while emphasizing that the audit applies to a declared quotient and a local linear model. Its reproducibility and ethics statements describe the released computational artifacts and the need for expert input when constructing real semantic fibers.

8 Conclusion

  • High current exposure with low intrinsic exposure calls for recalibration or a least-change projection; low intrinsic exposure with a large invariant-reader norm calls for regularization; high intrinsic exposure places repair on the representation.The audit also notes that aggregate spectra do not show where exposure sits, and that total cross-talk has a floor no readout optimization crosses.Source: §8 · p. 33
  • The triage applies wherever an explicit score reads a shared representation, but the real multilingual-encoder run covers one model, one layer, one score, and one language family.The real run falls in the conditioning regime; a sparse-autoencoder audit with distinct read and write directions is identified as a next measurement.Source: §8 · p. 33

Scope

  • The analysis treats one scored layer as a local linear model and takes the equivalence relation from an external scientific or governance judgment.The synthetic study fixes dictionaries and heads by construction and draws contrasts symmetrically, with rates anchored to one operating point.Source: Scope · pp. 33–34

Reproducibility Statement; Ethics Statement

  • A seeded CPU float64 command regenerates the paper's tables, figures, and numerical macros, while the optional real-encoder pilot has a separate pinned-model script.The work uses public benign text, involves no human subjects or personal data, and recommends domain experts and native reviewers for language- or culture-based fibers.Source: Reproducibility Statement · p. 34, Ethics Statement · pp. 34–35

A Numerical protocol

The numerical protocol constructs matched synthetic safety benchmarks across three geometries and separately stresses the untied control interface. It uses shared sample-level scores across the matched families and evaluates three-control compositions on held-out states and targets.

A Numerical protocol

  • The matched benchmark uses canonical safety writer e0 and block columns βe0 + γqj, giving the deployed tied reader exposure β|B| in every family.Stable frames use orthonormal qj, ill-conditioned frames make the final qj nearly dependent while retaining full column rank, and intrinsic frames use antipodal pairs; each geometry receives an independent Haar rotation.Source: A Numerical protocol · p. 41
  • Across matched families, hidden states combine label-scaled safety signal and isotropic noise, while block coefficients are Gaussian and endpoint residuals have scale 0.10.Noise is generated before family-specific rotation, matching deployed scores sample by sample across families.Source: A Numerical protocol · p. 41
  • The untied generator uses eight systems with calibrated readers and adds a feature-specific nonlinear leakage term to each linear setter.Pairwise residuals use 96 calibration states; the held-out task composes three controls in forward and reverse schedules on 160 new states and targets.Source: A Numerical protocol · p. 41

B Additional figures

The additional figures report validation regularization behavior, finite-sample exposure estimates, tied-map recovery, and a reader-optimized capacity floor. Their captions distinguish effects of geometry and leverage from effects of sample size.

B Additional figures

  • The stable family approaches exact invariance at moderate reader norm, while the ill-conditioned family requires rapidly increasing norm and validation selects partial repair.For the intrinsic family, validation returns the deployed reader.Source: Figure 12 · p. 41
  • The intrinsic family's exposure ratio remains 1.000 regardless of audit sample size, while the figure presents finite-sample learning curves.The caption characterizes additional audit pairs as improving estimation, not geometry.Source: Figure 13 · p. 42
  • Tied-map order-swap recovery matches the closed form for 48 distinct squared block coherences to 4.41 × 10−16.The figure is described as a consistency check of the ideal tied map.Source: Figure 14 · p. 42
  • Equal-leverage frames attain the reader-optimized capacity floor exactly, and the benefit over tied readers grows with leverage unevenness.The figure concerns the capacity floor of theorem 12(b).Source: Figure 15 · p. 43

Related papers

Loading 2609.14861v1…