PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Causal Masks: Auditing and Localizing Hidden Future-Token Leakage in Adaptive Sequence Models

Huiting Liu, Jinsong Wu

arXiv:2610.31522v1cs.AI

TL;DR

Hidden future-token leakage can make an autoregressive model appear better under teacher-forced evaluation even when generation deteriorates. The paper audits an adaptive residual-routing decoder using prefix-invariance checks, discrepancy and behavioral metrics, and layer-wise localization. In the CDR-AttnRes case, causal repairs removed detectable leakage and the large apparent advantage.

  • Problem

    Auxiliary operations can let future tokens influence prefix outputs despite a correct causal attention mask, making teacher-forced gains potentially misleading for autoregressive prediction.

    Source: §1 · p. 2

  • Method

    The paper applies the established prefix-invariance criterion, measures maximum prefix-output discrepancy with LeakScore and complementary metrics, and uses layer-wise profiles and controlled checks to interpret leakage.

    Source: §4.1 · p. 6, §4.2 · p. 7, §6 · p. 13

  • Results

    The CDR-AttnRes case showed a large apparent gain with detectable leakage and worse generation; two independently trained 12-layer repairs removed detectable leakage and returned validation loss and reference perplexity to the control scale.

    Source: §9 · p. 27

  • Takeaways & Limitations

    The documented gain was not a valid autoregressive improvement, and causal checks should be considered alongside teacher-forced results and free-running evaluation.

    Source: §9 · pp. 27–28

    The prefix-invariance audit is a practical check, not a formal proof of causality; synthetic experiments are controlled behavioral audits rather than task-level benchmark evaluations.

    Source: §8.2 · pp. 26–27

Abstract

from arXiv · show

Hidden future-token leakage can create false architectural gains: an autoregressive model may appear substantially better under teacher-forced evaluation while its free-running generation deteriorates. We investigate this failure in an adaptive residual-routing decoder whose sequence-level routing summary exposes prefix positions to future tokens despite a correct causal attention mask. An anomalous 24-layer checkpoint reduces validation loss from the baseline's 3.5958 to 1.4259, yet it fails a prefix-invariance audit and has substantially higher reference-model perplexity during generation. Layer-wise diagnostics and implementation inspection trace the violation to a full-sequence summary broadcast through the dynamic routing-query path. Two independently trained 12-layer causal repairs eliminate detectable leakage and return validation loss and reference perplexity to the control scale, removing the large apparent advantage. As diagnostic tools, we use the established prefix-invariance contract, maximum prefix-output discrepancy (LeakScore), complementary behavioral metrics, and layer-wise profiles. Supporting experiments cover six synthetic implementation pairs spanning five module families, clean pretrained controls, controlled fault injection, and cache consistency across numerical precisions. These results illustrate how hidden future-token access can produce misleading claims of architectural progress and why lower teacher-forced loss should be interpreted together with whole-model causal checks and free-running evaluation.

1. Introduction

A correct causal attention mask does not guarantee causal outputs: auxiliary operations can expose prefix positions to future tokens and create false architectural gains. The paper motivates whole-computation audits with a residual-routing case and diagnostic tools for detecting and localizing leakage.

1. Introduction

  • Causal self-attention restricts position t to x1:t, but pooling, temporal convolution, state aggregation, routing, or cross-time normalization can add non-causal paths.Prior work on linear attention and recurrent-style sequence compression likewise makes autoregressive behavior depend on prefix-consistent state aggregation.Source: §1 · p. 2
  • Hidden future-token leakage is suffix influence on prefix outputs through an auxiliary operation despite a correct attention mask, and can lower teacher-forced loss without improving valid autoregressive prediction.The resulting apparent improvement may depend on information unavailable during causal generation.Source: §1 · p. 2
  • The CDR-AttnRes follow-up showed an apparent teacher-forced gain alongside detectable leakage and worse free-running generation, which disappeared after causal repair.A full-sequence summary was broadcast as a low-rank correction to the depth-routing query; independently trained 12-layer repairs removed detectable leakage and the large apparent advantage.Source: §1 · pp. 2–4
  • The diagnostic framework compares shared-prefix outputs after suffix changes, uses maximum prefix-output discrepancy (LeakScore) and behavioral metrics, and inspects layer-wise profiles.Controlled module comparisons and pretrained-model checks support interpretation of these diagnostics.Source: §1 · pp. 2–3
  • The contributions include cross-module diagnostic evidence and layer-wise localization, with FirstLeakLayer identifying the first monitored layer showing detectable suffix influence.The paper presents these as diagnostic tools rather than a new causal contract.Source: §1 · pp. 3–4

2. Related Work

Related work addresses leakage at the dataset or evaluation-protocol level, causal sequence mixing and adaptive computation, and tools for behavioral testing or formal contracts. This paper focuses on internal future-to-prefix dependencies and a documented residual-routing case.

2. Related Work

  • Prior leakage concerns include dataset-level empirical reliability and benchmark overlap with training data [6], whereas this work examines future-token influence on prefix computation despite correct splits and attention masks.The focus is an internal dependency violation rather than dataset or protocol leakage.Source: §2 · p. 4
  • Existing causality-related contracts include prefix-invariance and cache-consistency checks, while formal frameworks address prefix preservation, cache alignment, and scan consistency [9].The paper's empirical emphasis is a false gain that disappears under causal repair, supported by cross-module diagnosis and layer-wise localization.Source: §2 · p. 4
  • Causal sequence models use varied mechanisms, including masked attention, linear attention and recurrent aggregation, causal convolutions [13], and structured or selective state updates [3].The cited architectures motivate checking temporal mixing and state aggregation; the paper does not claim they are non-causal.Source: §2 · p. 4
  • Routing can make token assignments depend on other sequence positions: expert-choice routing ranks across positions, while other work addresses causal routing [22] and computation allocation [2].This makes the information available to routing decisions an implementation concern.Source: §2 · p. 5
  • Behavioral testing and formal contracts provide related tools, but this audit requires position-wise outputs and layer-wise localization additionally requires monitored hidden states.Compatibility with arbitrary checkpoints is not assumed; the case concerns the full-sequence query correction added to the CDR implementation [5].Source: §2 · p. 5

3. Problem Definition

The problem is whether a sequence model's complete computation preserves prefix outputs when only the suffix changes. Hidden future-token leakage violates this condition through auxiliary operations even when masked attention is causal.

3. Problem Definition

  • Causal correctness requires identical outputs on a shared prefix whenever two input sequences share that prefix, and applies to the complete computation, including auxiliary operations.The model is considered on an input sequence x = (x1, x2, . . . , xT).Source: §3 · p. 5
  • Hidden future-token leakage is a violation mediated by an auxiliary operation outside masked self-attention while the attention mask remains causal.Unlike ordinary train/test leakage or benchmark contamination [6], it is an intra-model causal violation that can occur with a correct data split.Source: §3 · pp. 5–6
  • The audit examines which auxiliary operations violate prefix invariance, how strongly suffix perturbations affect outputs, and where that influence first becomes detectable.It uses LeakScore alongside complementary behavioral metrics and layer-wise profiles.Source: §3 · p. 6

4. Audit Methodology

The audit tests causal behavior by holding a prefix fixed and changing only its suffix, then checking whether prefix outputs remain unchanged within numerical tolerance. LeakScore supplies the maximum-discrepancy verdict, while complementary metrics, calibrated thresholds, and generation diagnostics characterize the audit and its evaluation context.

4.1. Prefix-Invariance Criterion

  • A causally correct model should keep shared-prefix outputs unchanged, up to numerical tolerance, when only the suffix changes.With other sources of variation controlled, this input-output test applies across sequence modules regardless of implementation.Source: §4.1 · p. 6

4.2. LeakScore

  • LeakScore is the largest absolute output deviation across prefix positions and output coordinates, with vocabulary logits used for language models.Mean diff, top-1 flip rate, affected-token rate, normalized maximum, and mean JS divergence characterize discrepancy but do not replace the absolute-tolerance verdict.Source: §4.2 · p. 7

4.3. Audit Procedure

  • The audit evaluates two same-prefix, different-suffix sequences under matched settings, compares outputs through the prefix, and fails when LeakScore exceeds ϵ.Dropout is disabled and other stochastic effects are controlled so input perturbation is the intended source of variation.Source: §4.3 · pp. 7–8

4.4. Tolerance and Numerical Robustness

  • 10^-5 is the default threshold, and conclusions remain unchanged across 10^-6, 10^-5, and 10^-4, separating repaired variants near zero from non-causal variants far above this range.The default is appropriate for the toy auditing setup in standard float32 evaluation.Source: §4.4 · p. 9

4.5. Generation Diagnostics

  • Generation diagnostics include fixed-GPT-2-medium reference-model perplexity, sampling entropy, Distinct-2, and Repeat-3, each computed on saved prompt-inclusive text.Reference perplexity is model-dependent, not a direct measure of semantic coherence or human evaluation; bootstrap intervals quantify uncertainty in its prompt-averaged estimate.Source: §4.5 · p. 9

5. Synthetic Auditing Experiments

The synthetic audits test whether auxiliary sequence modules preserve prefix invariance, separating causal validity from learned language-model quality. Across the tested module pairs, non-causal implementations leaked while their causal repairs did not, and robustness checks preserved that distinction.

5.2. Audited Modules

  • Six implementation pairs covered five module families: pooling, temporal convolution, linear attention, dynamic routing, and cross-time normalization.Pooling contributed two pairs; each pair included a non-causal implementation and a causally repaired counterpart.Source: §5.1 · p. 10

5.1. Experimental Design

  • The unaided, randomly initialized modules were tested on shared prefixes paired with independently sampled suffixes, using LeakScore tolerance ϵ = 10^-5.The audit tests prefix invariance rather than learned language-model quality.Source: §5.1 · p. 10, §5 · p. 10

5.3. Main Findings and Interpretation

  • All six non-causal implementations failed the audit, while their paired causal repairs had zero discrepancy under the same conditions.The tested failures spanned summary broadcast, temporal mixing, full-sequence KV aggregation, routing, and normalization; LeakScore values across modules are not a severity ranking because output scales differ.Source: §5.2 · p. 10, §5.3 · pp. 10–11, Table 1 · p. 11

5.4. Robustness Checks

  • Across prefix lengths 8, 16, 32, and 64, every non-causal module remained above tolerance and every causal repair remained at zero.This check used sequence length 80, unlike Table 1’s sequence length 32.Source: Table 2 · p. 12, §5.4 · p. 12
  • Across three seeds, all non-causal modules failed in 0/3 runs and every causal repair passed in 3/3; the same classification held at all three tested tolerances.The tested tolerances were 10^-6, 10^-5, and 10^-4; the weakest non-causal signal remained above 10^-4.Source: §5.4 · p. 12, Table 3 · p. 13

6. Real-Model Auditing

Audits on pretrained models test whether the prefix-invariance audit remains useful beyond synthetic modules, including under fault injection, different runtime paths and precisions, and layer-wise localization. The results show strong detection of injected leakage, while cache discrepancies in clean controls depend on precision and localization has a limited validation scope.

6.1. Experimental Setup

  • Each model was tested on 72 token-ID suffix-perturbed pairs (T = 64; prefixes 16, 32, and 48); leakage audits used float32 with dropout disabled and ϵ = 10−5.The audit reported max and mean prefix-output discrepancy, top-1 flip rate, and affected-token rate.Source: §6.1 · pp. 13–14

6.2. Clean Negative Controls

  • GPT-2, Qwen2.5-0.5B, and Mamba-130M passed all 72 tested pairs in float32 at ϵ = 10−5, with zero maximum and mean prefix-logit discrepancy and zero top-1 flips.These same-path clean controls support the tolerance for this protocol, not a universal threshold across runtime paths or precisions.Source: §6.2 · p. 14

6.3. Fault Injection and Sensitivity

  • Pass rate fell from 1.0 to 0.0 for every tested non-zero injection; affected-token rate stayed at or above 0.993, and Qwen max discrepancy rose with α.For Qwen, max discrepancy was 1.6 × 10−3, 7.9 × 10−3, and 1.6 × 10−2 across the tested strengths.Source: §6.3 · p. 14, Table 4 · p. 15

6.4. Cache and Precision Consistency

  • Reduced precision widened clean cache-path discrepancies: bf16 max discrepancy reached 4.0 for GPT-2 and 0.5 for Qwen2.5-0.5B; top-1 changed only for GPT-2 bf16 (1 of 32 steps).The authors recommend calibrating behavioral thresholds to trusted controls for each runtime and precision setting.Source: Table 5 · p. 15, §6.4 · pp. 15–16

6.5. Layer-wise Leakage Localization

  • Layer profiles were checked on one representative pair per model (T = 64, t = 32), but this was not an independent hold-out and its hidden-state intervention discrepancies are not comparable to output-logit wrapper results.The threshold is calibrated from clean-control discrepancies under the same localization protocol.Source: §6.5 · pp. 16–17, Table 6 · p. 17
  • Layer 3 was recovered as the first leak layer in all three tested models; clean models stayed below threshold, and final-layer discrepancy exceeded the injection-layer discrepancy.The layer-wise profile is zero before layer 3, then propagates through later layers with local increases and decreases.Source: Table 6 · p. 17, §6.5 · p. 17, Figure 3 · p. 18

7. Case Study: False Architectural Gains in CDR-AttnRes

The CDR-AttnRes case links anomalously low teacher-forced loss to detectable future-token leakage and degraded generation; causal repairs remove the leakage and apparent advantage. Audits, layer-wise profiles, and implementation inspection trace the issue to sequence-level summaries in the dynamic routing-query path.

7.1. Anomalous Training and Validation Behavior

  • Validation loss fell from 3.5958 for the baseline to 1.4259 for the pre-fix CDR-last 24L checkpoint, prompting a causal audit.Several pre-fix sequence-pooling variants also had much lower validation losses than trusted controls without correspondingly credible autoregressive behavior.Source: §7.1 · p. 19

7.2. Direct Audit with Prefix Invariance and LeakScore

  • The pre-fix 24-layer checkpoint failed prefix invariance, with maximum prefix-output difference 34.22 and mean per-pair top-1 flip rate 95.1%.The audit compared 72 input pairs with identical prefixes and different suffixes, using tolerance ϵ = 10−5.Source: §7.2 · p. 19

7.3. Layer-wise Localization of the CDR Leak

  • Every trusted control had no detected first leak layer, whereas every reported pre-fix variant began leaking at layer 0.The representative localization pair was included among calibration inputs, so the profiles were not an independent hold-out evaluation.Source: §7.3 · p. 20, Table 7 · p. 20

7.4. Generation Diagnostics of Controls and Leaking Checkpoints

  • Trusted controls had reference perplexity 16.8–19.8, while pre-fix variants had 198.5–700.5, with no overlap.The pre-fix 24-layer variant had the highest Distinct-2 (0.849) and lowest Repeat-3 (0.086), yet also the highest reference perplexity (700.5); lexical diversity alone did not establish generation quality.Source: §7.4 · pp. 21–22, Table 8 · p. 21

7.5. Root Cause: Sequence-Level Pooling in Dynamic Query Construction

  • A full-sequence summary was projected into a shared dynamic routing-query correction and broadcast to every token position, exposing prefixes to suffix information.The correction controls depth-wise routing, distinct from token-to-token masked self-attention.Source: §7.5 · p. 22

7.6. Repair Validation

  • Both independently trained 12-layer repairs pass the audit, return generation perplexity to the control scale, and remove the large apparent advantage.Their reference perplexities are 15.3 and 18.1, versus 16.8–19.8 for controls and 198.5–700.5 for leaking variants.Source: Table 9 · p. 23, §7.6 · p. 23

7.7. Quantitative Consistency Between Leakage and Apparent Gain

  • Across nine core checkpoints, lower validation loss was associated with greater LeakScore (Pearson r = −0.868), but not significantly within the four leaking checkpoints alone.The small, related checkpoint set and differences in pooling mode and depth preclude a dose–response interpretation; it does not establish that leakage magnitude determines validation loss.Source: §7.7 · pp. 23, 25, Table 10 · p. 25

7.8. Implications of the Case Study

  • The case supports auditing the complete computation before interpreting lower teacher-forced loss as architectural progress; layer-wise localization can guide implementation inspection.Independently trained causal repairs can test whether the apparent advantage persists under a valid dependency structure.Source: §7.8 · p. 25

8. Guidelines and Limitations

Causal checks must cover every operation that contributes to an output, not only masked attention. The paper’s evidence remains bounded by the audit’s practical scope, representative module coverage, simplified synthetic experiments, and a single CDR-AttnRes case.

8.1. Causality-Safe Design Guidelines

  • Every summary, state update, routing decision, and normalization statistic contributing to position i must respect its prefix boundary; a causal attention mask cannot compensate for future access elsewhere.Concrete designs include token-local or prefix-only summaries, left-padded convolution, prefix-KV aggregation, and prefix-consistent routing and normalization.Source: §8.1 · p. 26

8.2. Limitations

  • Passing prefix-invariance tests is evidence only against detectable suffix-to-prefix influence under tested perturbations and tolerance, not formal proof of causality.Attributing a detected violation to a hidden auxiliary path requires implementation or localization evidence.Source: §8.2 · p. 26
  • Coverage is not exhaustive across state-space models, mixture-of-experts designs, or hybrid architectures combining routing, compression, recurrence, retrieval, and state updates.Some variants may be causally safe by construction, while others may have subtler non-causal paths.Source: §8.2 · p. 26
  • The simplified synthetic experiments are controlled behavioral audits, not comprehensive task-level comparisons across datasets and training regimes.Their purpose is to test whether a module can leak future information.Source: §8.2 · p. 27
  • The CDR-AttnRes study is one case from the authors’ own architectural exploration, not evidence that every adaptive sequence model is affected.Independent codebases and public implementations would strengthen the empirical scope.Source: §8.2 · p. 27

9. Conclusion

The CDR-AttnRes case shows that hidden future-token access can create an apparent architectural gain despite a correct causal attention mask. The paper’s diagnostics and repairs support checking complete dependency structure alongside teacher-forced loss.

9. Conclusion

  • Two independently trained 12-layer causal repairs remove detectable leakage, return validation loss and reference perplexity to the control scale, and remove the apparent advantage.The paper concludes that the original result is not evidence of a valid autoregressive improvement.Source: §9 · p. 27
  • The paper uses prefix invariance, LeakScore, complementary metrics, and FirstLeakLayer, with supporting experiments across six synthetic implementation pairs and controlled pretrained-model checks.These results support interpretation under tested conditions, not proof of causality for all inputs or implementations.Source: §9 · p. 27
  • Check complete dependency structure before treating lower teacher-forced loss as architectural progress; combine causal checks with free-running evaluation, root-cause inspection, and repair validation.Pooling, temporal mixing, state aggregation, routing, and cross-time normalization can introduce future access outside masked attention.Source: §9 · pp. 27–28

Appendix A.1. CDR Training Configuration

The CDR variants use a shared training configuration and hyperparameters across 12- and 24-layer models. CDR adds low-rank projections for dynamic routing-query corrections in residual aggregation paths.

Appendix A.1. CDR Training Configuration

  • The 12-layer and 24-layer CDR variants use the same shared training configuration and hyperparameters.Source: Appendix A.1 · p. 28
  • Parameter counts are unique trainable parameters, with the shared input-embedding and output-head weight matrix counted once.Source: Table A.1 · p. 28
  • CDR adds rank-r down- and up-projections for dynamic routing-query corrections before both attention and MLP sublayers.Source: Table A.1 · p. 28

Appendix A.2. Auditing Protocols

The appendix specifies audit conditions, presents bounded illustrative generation excerpts, and describes supplementary materials and access requirements. It also records the paper’s funding sources.

Appendix A.2. Auditing Protocols

  • Public-model audits follow Section 6, while CDR audit and generation conditions follow Section 7, with metric definitions in Section 4; the nine core checkpoints use training seed 1337.Source: Appendix A.2 · pp. 28–29

Appendix A.3. Representative Generation Samples

  • The excerpts show the first 80 whitespace-delimited units from prompt 0 for three 12-layer checkpoints and do not establish factual accuracy or replace aggregate diagnostics.Full outputs for all 50 prompts, including 24-layer variants, are supplied in the supplementary materials.Source: Appendix A.3 · p. 29

Funding

  • The work was supported by grants from the China Guangxi Science and Technology Plan Project and two Chilean CONICYT programs.The listed grants are AD23026096, 1181809, and ID16I10466.Source: Funding · p. 30

Data and Code Availability

  • Supplementary materials provide experiment results, prompts, continuations, audit and generation code, and checkpoint mappings, but not model checkpoints because of storage constraints.Reproduction requires the relevant weights and, for retraining, preprocessed OpenWebText data and the training environment; exact numerical agreement across environments is not guaranteed.Source: Appendix A.2 · p. 29, Data and Code Availability · p. 30

Related papers

Loading 2610.31522v1…