PaperlayerView exact revision on arXiv

Source-linked AI summary

SCIT: Testing Causal Cache Carriers in Latent Chain-of-Thought Models

Yi Ding, Lijun Huang, Menglin Yang

arXiv:2608.27265v1cs.CL

TL;DR

Latent chain-of-thought hides the transformer object responsible for intermediate computation. SCIT uses exact counterfactual cache interventions and layered controls to identify that object. It finds a checkpoint-specific late value-cache suffix mechanism in CODI-GPT2, while broader cells show competence-gated carrier shifts rather than a universal latent-tail carrier.

  • Problem

    Latent chain-of-thought moves reasoning into continuous states, but latent-step interventions do not identify which transformer object carries the causal effect.

  • Method

    SCIT constructs exact source–recipient counterfactuals, patches cache segments, and combines component, hidden-state, semantic-source, decoded, and matched-corruption tests under competence gating.

  • Results

    SCIT finds complete sufficiency-and-necessity evidence for a late value-cache suffix mechanism in the main CODI-GPT2 checkpoint, while other cells show carrier-regime shifts.

  • Takeaways & Limitations

    SCIT supports a checkpoint-specific GPT-2 arithmetic mechanism and a competence-gated carrier map, not a universal latent-tail claim.

  • Takeaways & Limitations

    The evidence is limited by narrow synthetic arithmetic coverage and by failure to establish recipe-level replication across checkpoints and seeds.

Abstract

from arXiv · show

Latent chain-of-thought models move intermediate reasoning from emitted text into continuous states, improving compactness but hiding the causal object. We introduce SCIT, the Suffix Cache Interchange Test, a causal protocol that constructs exact source-recipient counterfactuals, patches declared cache segments, and identifies which transformer object carries the counterfactual computation. SCIT combines sufficiency tests with K/V component splits, hidden-state controls, semantic source controls, decoded validation, and matched corruption. On CODI-GPT2 and a Sim-CoT-style GPT-2 reproduction, counterfactual arithmetic transfers primarily through value-cache suffix trajectories rather than hidden states, keys, reusable answer slots, or single-token triggers. Complete sufficiency-and-necessity evidence for the late-value-suffix mechanism holds for the main CODI-GPT2 checkpoint; the Sim-CoT-style checkpoint shows the same sufficiency and decoded-control pattern but insufficient matched-corruption evidence for a necessity call. Beyond these local arithmetic cells, SCIT reveals carrier-regime shifts: arithmetic-like GPT-2/1B cells preserve latent-tail value/KV transfer, whereas competent 8B and repaired non-arithmetic cells route through prompt-prefix or full-cache K/V; boundary cells receive no mechanism call. SCIT therefore contributes a cache-level diagnostic, a checkpoint-specific GPT-2 arithmetic mechanism, and a competence-gated carrier map rather than a universal latent-tail claim.

1 Introduction

SCIT addresses the unresolved question of which transformer object carries latent counterfactual computation. It introduces a structured cache-carrier diagnosis with tiered claims spanning a checkpoint-specific mechanism and competence-gated regime shifts.

  • Motivation: Latent chain-of-thought moves reasoning into continuous states, making causal faithfulness harder to assess from readable rationales.The interpretability target becomes locating where hidden computation is implemented inside the transformer.
  • Problem: Existing latent-step interventions show whether latent computation matters but do not distinguish hidden states, keys, values, cache history, or answer-sharing sources.SCIT frames these alternatives as competing carrier hypotheses.
  • Contribution: SCIT constructs exact source–recipient counterfactuals, patches declared cache segments, and evaluates component, hidden-state, semantic-source, decoded, and corruption controls.Its output is a carrier map rather than a single positive patch score.
  • Results: The main CODI-GPT2 checkpoint supports complete sufficiency-and-necessity evidence for a late value-cache suffix mechanism.Counterfactual transfer is strongest in middle-to-late values, while hidden-only, key-only, one-token, same-answer, and partial-variable controls are weaker.
  • Results: Sim-CoT-style GPT-2 shows sufficiency and decoded-transfer support, but its matched-corruption evidence is insufficient for a full necessity call.The broader claim is diagnostic rather than universal.
  • Generalization: The paper reports a competence-gated carrier map: arithmetic-like GPT-2/1B cells retain latent-tail transfer, while competent 8B or repaired cells shift carriers.No-call boundary cells receive no mechanism interpretation.

2 Related Work

Related work makes latent computation manipulable and hidden representations inspectable, but SCIT targets causal localization of the transformer object carrying the computation.

  • Explicit reasoning traces: Explicit chain-of-thought provides readable traces, but plausible rationales are not necessarily faithful causal explanations.This motivates causal analysis beyond emitted reasoning text.
  • Latent and implicit chain-of-thought: Latent-CoT methods move reasoning into continuous states, while prior causal analyses treat latent steps as manipulable variables.SCIT complements these approaches by localizing the carrier object within the transformer.
  • Hidden-state inspection: Decoding methods can show that information is recoverable from hidden states, but recoverability alone does not establish that the model used those states.SCIT therefore treats decoding as a supporting check rather than the main causal test.
  • Causal localization: SCIT extends mechanistic-interpretability work on causal abstraction, interchange intervention, activation patching, path analysis, and circuit discovery.Its distinctive focus is cache-level carrier diagnosis for latent-CoT rollouts.

3 Method

SCIT patches defined cache objects in source–recipient rollouts, tests component and segment alternatives, validates decoded counterfactual transfer, and applies gated carrier decisions.

  • Protocol: The protocol defines a patch object, fixes an intervention grid, validates the readout, and applies a carrier decision rule.A mechanism call requires a specified carrier, closed control explanations, and competence gating.
  • Cache intervention: At a patch point, SCIT inserts selected source cache entries into corresponding recipient positions and optionally copies the source hidden state before continuing the rollout.The current hidden state is separate from the cache unless explicitly copied.
  • Cache object: A value-cache trajectory is an ordered suffix of value K/V entries, not a single residual direction, attention pattern, or decoded symbolic variable.Its sufficiency is tested by replacing the ordered value-cache slice.
  • Alignment: SCIT aligns variable-length source and recipient suffixes by reverse distance from their stopping points.Carrier calls are conditioned on the source and recipient stopping policy.
  • Intervention grid: The intervention grid varies latent-tail, prompt-prefix, and full-cache segments; K-only, V-only, and natural K/V patches; hidden-state coupling; layer blocks; and suffix length.Key-only and value-only splits are causal diagnostics, not ordinary activation states.
  • Protocol: SCIT combines a source cache slice with recipient-side fields and evaluates target win, margin, free generation, and matched corruption.Whole-cache replacement is an upper-bound control; segment calls use the smallest gated disjoint segment.
  • Controls: Synthetic arithmetic supplies exact counterfactuals, intermediate variables, random controls, and semantic controls for causal identification.Exact sources must transfer while same-answer, partial-variable, one-token, hidden-only, and key-only controls do not match the oracle effect.
  • Readout validation: Target win is the primary closed-set metric, supplemented by patched-minus-clean log-probability margin and decoded answer matching.Matched random corruption tests whether the recipient answer remains the winner after replacing selected entries.

4 Experimental Setup

The experiments evaluate SCIT across GPT-2 reasoning checkpoints, lexical variants, larger-scale cells, and boundary conditions using staged cache-interchange and control batteries. Reporting audits test whether competence and threshold choices alter carrier calls or computational cost.

  • Models and data: Experiments use CODI-GPT2 and a Sim-CoT-style GPT-2 reproduction across balls, books, and coins lexical variants.Both checkpoints use the same latent-cache interface and held-out source–recipient pairs.
  • Evaluation protocol: The experimental battery orders sufficiency, component and source controls, matched corruption, decoded validation, and carrier-regime calibration.Targeted repair is used only to satisfy the prerequisite that causal diagnostics follow task competence.
  • Efficiency and reporting: Competence sweeps at .70/.80/.90 and 16 carrier-threshold settings preserve all 12/12 audited calls, with exact and near ties at 0.063% and 0.072%.The audits support stability of the reported carrier map under these reporting choices.
  • Evaluation protocol: The full grid is a validation suite, while staged replay reduces oracle localization from 108 to 60 configurations overall.The replay recovers all 12 full-grid calls: 5 latent-tail and 7 prompt-prefix.
  • Efficiency and reporting: Naive staging changes runtime by at most 1.8%, whereas cached clean states reduce wall time by 34.6–40.0% on one matched 8B prompt-carrier cell.Peak CUDA allocation rises from 16.5 to 17.3 GiB, and the timing result is not an end-to-end or frontier-scale claim.

5 Results

SCIT identifies a middle-to-late value-cache suffix carrier in GPT-2 arithmetic, then maps shifts to prompt-prefix or full-cache K/V when competence, scale, or task conditions change. Boundary and training-path cells are withheld from localized mechanism claims when gates or evidence fail.

  • 5.1 Local arithmetic mechanism: CODI-GPT2 reaches 0.875–0.908 target win in final-step layers 8–9 values, while final layers 10–11 reach 0.017–0.029.Sim-CoT-GPT2 shows the same direction, with layers 8–9 at 0.958–0.992 and layers 10–11 at 0.133–0.192.
  • 5.1 Local arithmetic mechanism: A one-token suffix reaches only 0.203, whereas k ≥3 suffixes recover 0.984–1.000 target win in CODI-GPT2.Sim-CoT-GPT2 recovers earlier: k = 1 reaches 0.516 and k = 2 reaches 0.995.
  • 5.1 Local arithmetic mechanism: Matched corruption closes necessity for CODI-GPT2, dropping recipient target win to 0.211 with a 13.655 log-probability drop; Sim-CoT-GPT2 provides only directional necessity support.The local mechanism claim is therefore complete only for the main CODI-GPT2 checkpoint.
  • 5.2 Carrier-regime boundaries: Competent 8B arithmetic, entity, relation, and expression/list cells shift to prompt-prefix or full-cache K/V; main 8B rows show prompt-prefix K/V at 1.000 and latent-tail K/V at 0.000.Prompt-prefix sufficiency appears from the first scanned latent step, and visible fields remain readout-active after latent encoding.
  • 5.2 Carrier-regime boundaries: The relation-chain task reproduces a prompt-prefix carrier across three semantically equivalent renderings in every competence-passing cell.At both 1B and 8B, latent/prompt/whole-cache K/V target win is 0/1/1.
  • 5.3 Path sensitivity and reporting audits: Three newly trained CODI-style seeds receive no localized-carrier call because neither disjoint segment reaches the .80 carrier gate.Whole-cache transfer remains high at .870, .865, and .906, but late-value sufficiency is only 0.049–0.138.
  • 5.3 Path sensitivity and reporting audits: Sweeping competence and carrier thresholds leaves the same 5 latent-tail and 7 prompt-prefix calls, while exact top-score ties occur in only .063% of retained rows.These audits do not make the mechanism universal, but they rule out direct threshold and tie-breaking explanations for the displayed map.

6 Discussion

The discussion frames SCIT as a cache-level diagnostic whose strongest mechanism claim is checkpoint-specific, while broader evidence supports competence-gated carrier regimes rather than a universal latent-tail carrier. It also outlines controlled extensions for executable GSM8K subsets and recurrent architectures.

  • Scope of the mechanism claim: In the studied GPT-2 arithmetic checkpoints, the transferable object is a context-bound value-cache trajectory, not hidden state alone, key routing alone, or a reusable answer slot.The claim is based on controlled sufficiency and controls, not a decoded symbolic trace or named single-head circuit.
  • Scope of the mechanism claim: The 8B carrier shifts to visible prompt fields, which are already sufficient before latent rollout and remain readout-active after latent encoding.Latent iterations may perform readout or control operations over an already informative prompt-prefix cache.
  • Generalization boundary: The appropriate generalization target is a competence-gated carrier-regime map, not a universal carrier prediction for new tasks or models.The present evidence supports a focused GPT-2 arithmetic mechanism and a broader diagnostic framework.
  • Future extensions: A controlled GSM8K extension would use executable solution graphs, verified counterfactual labels, rendering variants, competence gates, decoded transfer, and matched-corruption controls.The proposal is an evaluation pathway rather than a result from the current experiments.
  • Future extensions: For recurrent or loop models, SCIT could map carriers by iteration or reverse distance from the realized stopping point and periodically remap them during training.The proposed training extension combines clean loss with interchange loss while preserving inactive recipient labels.

Limitations

The paper limits its claims to controlled synthetic arithmetic and tested checkpoints, while broader task, scale, stopping-policy, and decoding generalization remain unresolved.

  • Task breadth: The experiments use lexical variants of one two-hop arithmetic diagnostic rather than a broad benchmark suite.The authors caution that mechanisms found on synthetic two-hop arithmetic may differ from those used on natural GSM8K-like problems.
  • Stopping policy: The variable-stop pilot freezes stopping decisions, tests one synthetic checkpoint, and provides no matched-corruption necessity evidence.Its joint K/V+h interpretation is specific to the tested stopping policy, not evidence for learned or open-ended halting.
  • Checkpoint and scale scope: The OOD numeric-range check is secondary because its random-source baseline is high, while carrier localization shifts across checkpoints and scales.Public and matched LLaMA3-1B cells support value-cache transfer with a shifted region, whereas matched LLaMA3-8B supports full-cache/prompt-prefix transfer.
  • Replication scope: The strongest sufficiency-plus-necessity claim concerns one realized CODI-GPT2 checkpoint, not a stable property of the CODI training recipe.The Sim-CoT-style reproduction has strong sufficiency and decoded-control support but only directional matched-corruption damage; newly trained CODI-style seeds do not reproduce the late value carrier.
  • Generalization: Scale generalizability requires matched competence-gated diagnostics across architectures and sizes, and broader decoding and downstream behavior remain to be tested.The authors also caution that architecture-relative layer partitions are needed instead of a fixed GPT-2 notion of “late.”

Ethics Statement

The appendix documents the controlled experimental design, intervention definitions, arithmetic templates, scoring procedures, and evidence map used to evaluate SCIT and its carrier-regime calls.

  • Appendix organization: The appendix organizes supporting material by evidential role, including expanded schematics, intervention tables, carrier ledgers, and boundary classifications.No-call is a reporting category covering distinct competence, support, source-control, and high-baseline OOD reasons.
  • Arithmetic controls: Arithmetic templates encode the computation as P = A − C and answer P + B across balls, books, and coins surface variants.Controls separately preserve final answers, intermediates, partial variables, or surface decompositions to test semantic specificity.
  • Scoring and corruption: Localization scores fixed candidate answers with summed teacher-forced log-probabilities and uses ∆cf−rec for continuous-margin checks.Matched random sources are controlled by template family, numeric range, latent step, and cache-segment shape, excluding recipient, donor, and oracle answers.
  • Intervention design: SCIT interventions vary cache segment, K/V component, hidden-state handling, source semantics, decoded behavior, and matched corruption.Cache-only patches copy K/V entries while retaining the recipient hidden state; hidden-only patches copy the hidden state while leaving K/V unchanged.
  • Evidence scope: The appendix distinguishes core mechanism evidence from competence-gated calibration across arithmetic-like, scaled, repaired, and boundary cells.Tables summarize carrier regimes, prompt-prefix follow-ups, truncation probes, readout ablations, and no-call rows rather than broad benchmark wins.

A.5 Scope, Generalization, and Repair Boundaries

The controlled extensions define SCIT's scope: exact-counterfactual diagnostics trade task breadth for source-control validity, and competence-gated repairs reveal distinct carrier regimes rather than broad generalization. Arithmetic-like cells preserve latent-tail transfer, while competent larger or repaired cells can shift to prompt-prefix/full-cache mechanisms or remain boundary cases.

  • Scope and calibration: The main GPT-2 arithmetic result survives free decoding, while restricted layer localization is not stable across architectures.A clean three-hop arithmetic task preserves value-cache transfer, but non-arithmetic cells require competence before mechanism interpretation.
  • Repair boundaries: Widened-support repairs improve far-OOD deployment, but ultra-OOD cells fail the free-generation gate or remain source-control limited.The hard-cell replicate preserves prompt-prefix/full-cache damage under corruption and leaves expression-tree as a no-call cell at ultra-OOD distance.
  • Carrier shifts: Competent relation-chain cells reach target win 1.000 under oracle prompt-prefix, query-only, and whole-cache K/V patches, while latent-suffix K/V patches do not carry the queried-person binding.Full prefix/cache patches carry the complete source answer under mismatched fields.
  • Repair boundaries: The diagnostic separates no zero-shot breadth, insufficient 1B repair, and 8B repair with a prompt-prefix/full-cache K/V carrier.These regimes should not be read as universal task-solving results.
  • Scope and calibration: SCIT-Bench is a controlled causal diagnostic with exact source semantics, not a general reasoning benchmark.The construction trades task breadth for exact source–recipient interchange and valid negative controls.
  • Arithmetic carrier checks: CODI's arithmetic transfer uses a suffix trajectory rather than a single token, and ranked heads recover transfer faster than random same-size sets.The carrier is therefore temporally extended and sparse-distributed across value heads.

A.7 Carrier-Regime and Scale Details

Carrier identity depends on competence, task structure, and scale: arithmetic-like cells retain latent-tail value/KV transfer, whereas competent repaired or larger cells often use prompt-prefix or full-cache K/V. Calibration and localization tests delimit these calls rather than extending them universally.

  • Calibration: The calibration battery reports oracle transfer, semantic-source margins, free-generation success, and corruption-based necessity, with competence gating mechanism claims.Entity and graph rows that fail competence are excluded from mechanism interpretation.
  • Scale and carrier regimes: Augmented arithmetic, conditional, and relation cells are refreshed with N = 128 held-out pairs per task, while prompt-prefix and whole-cache patches can reach target win 1.000 in relation-chain cells.The matched 8B checkpoint has full-cache transfer on competent cells, but latent-tail K/V alone does not recover the source signal.
  • Scale and carrier regimes: Matched corruption separates regimes: 1B arithmetic-like and conditional cells depend on latent-tail value/KV, whereas 8B expression/list and matched 8B cells depend on prompt-prefix value/KV.This inverse necessity pattern supports a carrier-segment shift rather than sufficiency alone.
  • Layer localization: The strongest restricted GPT-2 localization is consistently layers 8–9, while the final 10–11 block is weak.Four-layer windows containing 8–9 nearly recover the full late-window effect.
  • Repaired non-arithmetic cells: Expression-tree and list-filter/sum repairs use visible prompt fields rather than latent-suffix or answer-prefix spans when they pass the gate.Expression-tree follows operand/fact bindings, while list-filter/sum follows rule/fact and threshold/filter fields.
  • OOD checks: The late-value pattern persists for larger OOD numeric values, strongest in CODI-GPT2, while Sim-CoT-GPT2 retains the direction with a larger nonlate baseline.The OOD semantic-control diagnostic is supplementary because its random-source baseline is high.

A.8 Component, Source, and Decoded-Behavior Details

Component, source, and decoding controls isolate the arithmetic carrier as value-cache suffix content rather than hidden states, keys, partial semantics, or a single token. Additional controls show decoded transfer, temporal suffix dependence, sparse-distributed heads, and boundary cases where apparent success reflects interpolation or lacks stable deployment evidence.

  • Component controls: Latent-cache-only replacement reaches 0.704/1.000, compared with 0.246/0.000 for hidden-state-only replacement at the final step.The Sim-CoT-GPT2 diagnostic gives the same qualitative conclusion.
  • Component controls: Key-only suffix patching fails to transfer the final-step counterfactual, whereas value-only suffix patching is sufficient.This component split localizes the transferable content to values rather than keys.
  • Source and decoded controls: Exact-context cross-template sources preserve oracle transfer, while partial-variable controls remain weak across both GPT-2 variants.Greedy and sampled decoding reproduce the same separation, with exact-context patches near oracle and partial controls near zero.
  • Generalization boundaries: The fixed-rule holdouts pass on seen supports but fail on held-out supports, while partial leave-one signals are consistent with local interpolation rather than algorithmic state updating.Matched 8B two-step success also fails shorter and longer horizon checks.
  • Trajectory structure: The operative object is a suffix trajectory: CODI requires a short cache window, and Sim-CoT benefits from more than one latent position.A single final-token trigger does not explain the transfer.
  • Necessity and interpretation: Sim-CoT's selective late-value corruption has the same sign as CODI-GPT2 but is weaker, so it provides directional rather than complete necessity evidence.The broader trajectory-content check likewise rejects a single final token or single value head as the explanation.
  • Head structure: Ranked value heads recover transfer faster than random same-size sets, supporting a sparse-distributed carrier rather than a named single-head circuit.The effect is not uniformly spread across all heads.

B Reproducibility Notes

The paper provides public code, configurations, generators, and an artifact builder for reproducing SCIT runs and regenerating tables.

  • Reproduction: The repository supports pair generation, localization and carrier-split runners, and markdown/CSV table regeneration.Exact command paths are publicly available.

C Compute

SCIT's minimum diagnostic runs are lightweight on a single GPU, while the full evidence suite adds substantially broader training and repair jobs. Staged execution with caching reduces wall time while preserving full-grid results.

  • Each held-out localization or semantic diagnostic run uses one GPU and typically finishes within tens of minutes to under one GPU-hour.This estimate applies to a two-seed GPT-2 template run at the current sample size.
  • 907.1 → 544.6 s and 822.9 → 538.1 s, reducing wall time by 40.0% and 34.6% with staged execution plus caching.The corresponding staged records match full-grid execution with no missing, extra, or error rows.
  • Naive two-process staging changes wall time by only −1.5% to +1.8%, attributing the observed speedup to staging plus caching rather than staging alone.
  • Caching 192 clean states raises peak CUDA allocation from 16.462 to 17.291 GiB.
Loading 2608.27265v1…